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ABSTRACT 

Encouraged by evidence linking quality early care/education 
programs and improved student outcomes, a variety of school readiness 
programs and initiatives have been created. This report reviews and 
synthesizes evaluation studies conducted on early childhood interventions, 
focusing on programs emphasizing a school readiness goal. The report provides 
an overview of the evidence regarding effects of model early care and 
education programs and describes the program characteristics associated with 
program effectiveness. An extensive, systematic search of the literature 
identified 32 impact evaluations of recent school readiness initiatives, with 
selection criteria used to identify 20 studies most relevant to the research 
questions. Results from 11 studies coded as pre-experimental or correlational 
were analyzed and reported separately from the 9 quasi-experimental or 
experimental studies. Based on the synthesis, it was concluded that wide- 
scale school readiness interventions can have moderate effects on child 
outcomes, with the strongest evidence for benefits to children's social- 
emotional development. Positive results were also reported for 
language/literacy, mathematical thinking, and physical/health development, 
outcomes such as better attendance or fewer referrals for special 
services once the child entered school. The report notes that it was 
difficult to discern whether specific program features were associated with 
positive participant outcomes because experimental and quasi-experimental 
designs typically did not examine these relationships. Implications for 
program evaluation relate to research design and instrumentation as well as 
to program quality and the classroom dynamics. The report's two appendices 
include the data coding sheeting and tables delineating program 
characteristics and design characteristics of quasi-experimental and 
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Evaluations of School Readiness Initiatives: What Are We Learning? 

Introduction 

The link between quality early care/education programs and improved student outcomes is fairly 
well established. Decades of research have clearly demonstrated that high-quality early 
care/education can promote positive developmental outcomes, both while children are in 
childcare and later as they enter school. Specifically, studies (e.g., Barnett, 1995; Guralnick, 
1997; Karoly, Greenwood, Everingham, Hoube, Kilbum, Rydell, Sanders, & Chiesa, 1998) have 
found that children who received high-quality early care/education perform better on measures of 
cognitive and language development, and they are less likely to have been retained in later 
grades of school than children who did not receive high-quality care. Moreover, high-quality 
care shows positive associations with early social and emotional development (e.g., fewer 
behavior problems, greater cooperation with adults and peers, more sophisticated play). These 
findings seem to be particularly true, for young children who are living in situations that place 
them at greater risk for school failure (e.g., poverty, low level of maternal education, single- 
parent household, and other factors that may limit their access to resources and experiences that 
enhance learning and development). Research indicates that these children are, in fact, the ones 
who benefit most from quality early childhood services. 

Encouraged by this evidence, a wide variety of school readiness programs and initiatives have 
been created. At the heart of these efforts is a belief that the early years constitute an especially 
sensitive time in children’s learning and development. Moreover, many experts believe that 
failure to provide children with the necessary resources and supports may create later problems 
that are far more difficult and costly to remedy. Head Start and Even Start represent examples of 
federal school readiness programs; however, a number of states and local districts have 
implemented school readiness programs as well. Examples include North Carolina’s Smart Start 
program, Michigan’s pre-kindergarten program, and Georgia’s lottery-funded school readiness 
program. 

The purpose of this paper is to conduct a review and synthesis of evaluation studies conducted on 
early childhood interventions, with a focus on programs that emphasize school readiness as a 
goal. This synthesis begins with an overview of the evidence regarding the effects of model early 
care and education programs and includes a discussion of the program characteristics associated 
with those findings. This literature provides an important context for understanding the actual 
subject of the synthesis, evaluations of school readiness initiatives. Following this section, the 
methodology and findings of the synthesis are presented. Implications of the findings are 
discussed in the final section. 

Previous Research on Early Care and Education Programs 
Model Early Care and Education Programs 

The positive relationship between high-quality early childcare programs and children’s 
development is one of the most consistent findings in developmental science (National Research 
Council, 2000). However, much of what is known and accepted about the effects of early 
childhood education programs comes from a small group of “model” programs. In this set of 
programs, researchers developed their own programs that allowed them to study the effects of 
exemplary practices. Typically, these programs received superior funding and were able to hire 
more highly qualified staff than typical early care and education programs. As a result, they were 
able to maintain a higher quality of care than large-scale, publicly funded programs (Barnett, 
1995). 
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A recent article by Currie (2001) highlights four specific studies of model programs that stand 
out due to the rigor of the research. Each of these studies randomly assigned children to either 
treatment or control groups. In addition, none of these studies suffered from excessive attrition, 
and researchers were able to follow children at least into middle school. They include the Perry 
Preschool Project, the Carolina Abecedarian Project, the Early Training Project, and the 
Milwaukee Project. (The Infant Health and Development Project also used a randomized design 
and had low attrition, but children were only followed until age eight). 

Follow-up studies of the Perry Preschool Project continue to gain attention because of the results 
and longevity of the study. The study originally involved 123 three- and four-year-olds. The 
children were randomly assigned to either a high-quality preschool program or a no-program 
control group. The children assigned to the preschool program attended a half-day program each 
day, and their families received a weekly home visit. Classroom ratios were very low (1:6), and 
all of the preschool teachers had master’s degrees and training in child development. Data have 
been collected on children from both the treatment and control groups— annudly from ages three 
through 11, ages 14-15, age 19, and age 27. Results indicated that participants in the preschool 
program had higher achievement test scores and graduation rates, as well as fewer incidents of 
criminal arrests and welfare use as reported by program participants (Schweinhart, Barnes, & 
Weikart, 1993). 

In the more recent Abecedarian Project, researchers randomly assigned 111 infants into a 
treatment group that received high-quality childcare and a control group (Campbell & Ramey, 
1994, 1995). The goal of the project was to prevent mental retardation and improve academic 
achievement when the children entered school. The children participated in their assigned groups 
for the first five years of their lives. Of the original 111 children, 92 were followed until age 15. 

At the end of the preschool intervention, there was a statistically significant IQ (seven points) 
difference between the treatment and control group (Campbell & Ramey, 1994). This difference 
was seen again at age eight and age 12. At age 15, although the children who received treatment 
continued to outscore children in the control group, the difference was no longer statistically 
significant. Other findings indicate that program participants continued to score significantly 
higher on reading and mathematics tests (at ages 8, 12, and 15). They also were less likely to 
have been retained (significantly different at ages 8, 12, and 15) or to have received special 
education services (age 15 only) (Campbell & Ramey, 1995). 

The Early Training Project began in 1962 with a group of low-income, African-American 
children (Gray, Ramsey, & Klaus, 1982). The purpose of the project was to assess the impact of 
a preschool intervention on attitudes relating to achievement and on academic performance. It 
also focused on engaging mothers in their children’s learning. The intervention program 
consisted of a ten-week summer preschool program for the two to three summers prior to the first 
grade, plus weekly home visits during the remainder of the year. Results indicated some positive 
effects on individual IQs for the first few years; however, this effect diminished over time. 
Greater impact was noted on school performance variables. For example, the number of children 
in special education was significantly smaller for the experimental group. There also was a small 
effect on the grade retention rate and an increase in high school graduation rates. In terms of teen 
pregnancy, although there was no difference between the treatment and control in the rate of 
pregnancy, girls in the treatment group were significantly more likely to return and complete 
high school than girls in the control group. 

The Milwaukee Project also was an attempt to improve cognitive development in high-risk 
families (Garber & Heber, 1977). This intensive intervention was designed for low-income, 
inner-city families with a maternal IQ below 75 and children between the ages of three months 
and six months. The six-year program offered extensive, daily, year-round educational 
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programming for the child and vocational/education services to families, including job training, 
remedial education, parent counseling, and family crisis intervention. 

Results of this program suggested that children who participated in the intervention obtained 
higher IQ scores, demonstrated better problem-solving skills, and had better attitudes toward 
learning. In addition, program participants scored significantly higher on language development 
than children who did not participate in the program. 

Although “model” programs have provided the strongest evidence that early childhood education 
programs can have substantial effects on children’s lives, additional evidence is available from 
large-scale research studies. 

Evidence from Large-Scale Studies 

The Cost Quality and Outcomes in Childcare Centers Study (Cost, Quality, and Outcomes Study 
Team, 1995) demonstrated that positive effects are not limited to special early intervention 
programs like the Abecedarian ftoject — even children in high-quality childcare programs 
benefit. The study, which began in 1993, was designed to examine the influence of care on 
children’s development during preschool and as they moved into elementary school. Initially, 
researchers recruited a stratified random sample of 401 childcare centers within four states: 
California, Colorado, Connecticut, and North Carolina. Within each center, two classrooms were 
randomly selected for observation. These observations documented the quality of services 
provided by the center and the range of costs associated with those services. 

After collecting data on the costs and quality of these programs, researchers identified 
classrooms that enrolled children who would be entering kindergarten in the fall of 1994. A total 
of 826 children were selected from 183 classrooms. Overall, the initial sample included 51% 
males and 49% females. Of these children, approximately 30% were children of color. The 
average level of maternal education was 14.2, and the average family income was $47, 753. 
Because of attrition, the number of children declined somewhat across each year so that by Year 
5, there were 418 children from 160 of the 183 classes still participating. 



Children were followed over a four-year period through the end of second grade. Controlling for 
demographic characteristics (i.e., maternal education and family income), researchers found that 
children in high-quality childcare programs performed better on measures of cognitive and social 
skills in childcare and at least through the end of second grade (the latest data point that has been 
reported). High-quality childcare seemed to have the greatest impact on children who are 
traditionally at risk of school failure (i.e., whose mothers had low education levels), and, 
conversely, when these children are enrolled in poor-quality childcare, they are at greater risk for 
doing poorly in school. 

Similar results were found in a study conducted by the National Institute of Child Health and 
Human Development (NICHD, 2001). In this study, researchers examined the effects of 
children’s experiences in childcare and their development over time. This study involved a total 
of 1364 children and their families recruited from ten sites located across the country. Beginning 
in 1991, children were enrolled at birth, and plans are to follow the children to the year 20^05 
through their sixth year of school. 

Families included in the sample vary significantly with regard to economic and ethnic 
backgrounds. Approximately 76% of the families are white of non-Hispanic origin, nearly 13% 
of families are black, 6% are of Hispanic origin, 1% are Asian/Pactfic Islanders/ American 
Indians, and 4% are other minorities. About 10% of the mothers had less than a 12*-grade 
education, slightly over 20% of the mothers had a high school diploma, one-third had some 
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college, 20% had a college degree, and 15% had a graduate or professional degree. The average 
income for families participating in the study was $37,781. Researchers used this information to 
focus on the unique contribution that childcare experiences make to children’s 
development— testing the effects of early care and education programs above and beyond the 
contributions made by the family and child characteristics. 

Although the study is continuing, findings to date suggest that children in high-quality childcare 
arrangements score higher on tests of cognitive and language ability than children in lower- 
quality care. Children whose environments were stimulating and well organized scored higher on 
tests of vocabulary, short-term memory, and attention. Moreover, children from these 
environments got along better with peers. 

Together, these findings demonstrate the importance of high-quality care as it relates to 
children’s intellectual development and preschool skills. 

Characteristics of High-Quality Early Childhood Programs 

Over time, early childhood research has shifted from “Can early childhood programs positively 
affect the development of children?” to the more refined question of “What are the specific 
features/elements that seem to generate these effects, and do variations of these features 
moderate the effect on children’s development?” 

According to Bryant and Maxwell (1996), determining the characteristics of successful programs 
can be very difficult because of the complexity of including all of the potential characteristics in 
a single study. However, several reviews of this literature seem to agree that there are certain 
aspects of program quality that lead to greater benefits for children (Barnett, 1995; Clarke- 
Stewart, 1987; Halpem & Weiss, 1990; Phillips & Howes, 1987; Ramey & Ramey, 1992). Two 
common terms recur in the conceptualizations of program quality: (1) structural factors and (2) 
process factors. Structural factors usually refers to the way in which the program is organized or 
structured and includes variables such as: 

Staff-to-child ratio 
Group sizes 

Specialized staff training in early childhood education 

Other features of program quality, called process factors, focus more on the relationships 
between children and staff and the interactions that occur around learning. Some of these 
features include having: 

Caring, responsive teachers who care for the same group of children over a long time period 
A curriculum that provides an opportunity to participate in a variety of developmentally 
appropriate activities 

Although it is the teacher and caregiver behaviors that form the core of children’s classroom 
experiences, many have speculated that it is the structural factors that provide the foundation that 
either hinders or enhances the quality of process in a program. That is, teachers are more likely 
to engage in appropriate caregiving practices if they are part of a program that maintains good 
structural factors, such as smaller group sizes and larger numbers of staff per children (Powell, 
1995). 



Need for School Readiness Services and Evidence of their Impact 

Despite the considerable evidence connecting quality early care and education with children’s 
success in school, many children continue to enter school unprepared. Data from the recent Early 
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Childhood Longitudinal Study, Kindergarten Class of 1998-1999 (ECLS-K) indicate that many 
children do not have the prerequisite skills and abilities they need to be successful in school 
(West, Denton, & Germino-Hausken, 2000; Zill & West, 2000). This study— one the largest and 
most comprehensive of its kind— was designed to describe the readiness of children as they 
entered into kindergarten. Researchers began by following a nationally representative sample of 
approximately 22,000 children from 1,277 programs. Data were collected along a number of 
dimensions of child development, including cognitive development, social/emotional 
development, and physical/psychomotor development. The results indicated that most of the 
children in the study entered kindergarten with basic skills pertaining to reading, mathematics, 
and social relationships. However, a significant percentage lagged behind their peers. For 
example, 18% of the children did not demonstrate that they were familiar with the conventions of 
print (i.e., they did not know that English print is read from left to right or where a story begins 
and ends), 34% could not identify letters of the alphabet by name, 42% were unable to count 20 
objects, and six percent could not count ten objects. 



Addressing the Need 

Given the apparent need to address children’s readiness and the evidence linking preschool 
participation with readiness, the prevalence of early childhood early intervention programs has 
increased dramatically over the past several years. In particular, states have begun to recognize 
the importance of early education in preparing children for school. A number of states and local 
districts are starting to integrate their pre-kindergarten initiatives with other early childhood 
programs— such as Head Start and childcare— to more effectively use the funds and resources 
that are available (Schulman, Blank, & Ewen, 1999). 

According to recent statistics {Education Week, January 2002), 39 states, plus the District of 
Columbia, have begun to offer their own classroom-based pre-kindergarten programs for at least 
some of their three- to-five-year-old children. This number is up from ten states in 1980. In 
addition, 21 states, plus the District of Columbia, provided substantial funds to supplement Head 
Start programs operating in their states. 

Despite these efforts to reach young children, most of these programs serve only a fraction of the 
children who could benefit from these services. Currently, most states focus their efforts on the 
neediest children— that is, children from low-income families or children with other risk factors, 
such as limited English proficiency or a teenage parent. Only three states — Georgia, New York, 
and Oklahoma— plus the District of Columbia have made preschool services available for all 
four-year-olds regardless of income of other identifiable risk factors {Education Week, January 
2002). And within these, only Georgia mandates the provision of pre-kindergarten services in all 
districts. New York and Oklahoma allow districts to choose whether or not they will use state 
funds to provide pre-kindergarten services. 

Even in these states where pre-kindergarten services are open to all who are interested, only a 
portion of the children are served. Georgia’s program serves the greatest number of four-year- 
olds with approximately 63,000 enrolled. When combined with Head Start enrollment, this 
figure translates into about 70% of four-year-olds in some form of publicly funded preschool. In 
contrast to Georgia, Oklahoma only serves about one-half of its four-year-olds, and New Y ork 
serves approximately one-quarter {Education Week, January 2002). 

Beyond these state efforts, a number of local communities and districts are making a 
commitment to early childhood education. According to a recent report released by the General 
Accounting Office (GAO, 2000), local school districts are beginning to apply some of their Title 
I funds toward preschool services. A survey of 16,000 school districts from across the nation 
revealed that approximately 17% of the districts receiving Title I funds used a portion of those 
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funds for preschool services during the 1999-2000 school year. Numbers indicated that 
approximately 313,000 preschool-age children were served with those funds— this represents 
about eight percent of the children who will ultimately enter public school kindergarten. 

Evaluations of School Readiness Initiatives 

At the same time that programs and agencies are increasing their investments in children’s early 
education, there also is increasing pressure to provide evidence of the effectiveness of these 
programs. Although much has been written about the effectiveness of small-scale model 
programs— and this literature has contributed much to our understanding of the benefits of early 
childhood intervention — there is little evidence to document whether these large-scale, publicly 
funded programs actually help children perform better in school. A recent survey of state-funded 
preschool programs (Ripple, Gilliam, Chanana, & Zigler, 1999) found that less than half of these 
programs had conducted an evaluation of their program’s effectiveness. This raises numerous 
questions about the effectiveness of these programs and their capability of producing significant 
benefits. 

In an attempt to answer some of these questions, Gilliam and Zigler (2001) conducted a meta- 
analysis of the available evaluations of state-funded preschool programs from 1977 to 1998. The 
authors concluded that, despite some of the methodological limitations of the evaluations, state- 
funded preschool programs might indeed help children perform better when they enter school. 
However, since this study was conducted, other states have implemented preschool programs, 
and additional work is needed to determine their effects. 

As a result, the purpose of this section of SERVE’ s Expanded Learning Opportunities (ELO) 
research synthesis is to examine the effectiveness of a particular type of early care and education: 
school readiness initiatives. Rather than looking globally at the relationship between quality 
early childhood programs (which can include a variety of programs such as childcare, preschool, 
etc.) and student outcomes, the synthesis will focus on programs and initiatives designed 
specifically to prepare children for success in school. The synthesis will examine data that have 
been collected on these programs to answer the following questions: 

What types of school readiness initiatives have been evaluated, and what research 
methodologies have been used to collect the data? 

What are the outcomes associated with participation in school readiness programs? 

What practices seem to best prepare children to enter school ready to learn and succeed? 

By focusing on these questions, it is hoped that this synthesis will provide a detailed, critical 
review of the methods and findings of school readiness evaluations conducted recently and that it 
will clarify recommendations for policy, practices, and program evaluation. 

The next section provides a detailed description of how the literature review was carried out and 
the process for including or excluding studies in the synthesis. 

Methodology 



Identifying the Relevant Literature 

The Literature Search. Following a process outlined by Cooper (1998), an extensive 
search of the literature was conducted to identify impact evaluations of recent school-readiness 
initiatives. To identify relevant articles and reports, a list of key words was compiled that might 
be used to describe school readiness initiatives. Search terms included; early intervention, school 
readiness. Head Start, family literacy. Even Start, pre-kindergarten, and kindergarten readiness. 
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These terms were combined with key words such as outcomes, evaluation, and results to search 
specifically for studies that relate to program evaluations and child outcomes. 

The search process included both formal and informal channels of research information. 
Databases were searched, including ERIC, PsychINFO, Dissertation Abstracts International, and 
SSCI. In addition, an extensive search was conducted of the World Wide Web using the terms 
outlined above. This led investigators to a variety of websites including: Child Trends, National 
Center for Early Development & Learning, Mathematica, RAND, Southern Regional Education 
Board, U.S. Department of Health and Human Services, and Regional Educational Laboratory 
websites. Links from each of these sites were followed and searched thoroughly. Finally, 
relevant conference presentations were located by reviewing past conference programs and 
proceedings for the American Educational Research Association (AERA), Association for 
Childhood Education International (ACEI), Head Start Research Conference, Improving 
America’s School Act (lASA) Conferences, National Association for Elementary School 
Principals (NAESP), National Head Start Research Conference, and Society for Research in 
Child Development (SRCD). 

In addition to the literature review process, early childhood specialists in each state department 
of education were contacted regarding their state’s pre-kindergarten program. Specialists were 
contacted via meetings, listservs, and emails and asked to provide any evaluation reports that 
might document the effectiveness of their program. 

The search process was conducted between January and April 2002 and yielded a total of 53 
articles, reports, conference presentations, and dissertations disseminated between January 1997 
and April 2002. Once these reports were compiled, the research team reviewed each of the 
documents to determine its suitability for possible inclusion in the synthesis. Documents that 
failed to provide a description of a specific program or initiative or documents that did not report 
on child outcomes were immediately eliminated. A total of 32 documents were identified as 
possible candidates for inclusion in the synthesis. 

Narrowing the Pool. Selection criteria for the synthesis defined the nature of the program 
being evaluated and some basic requirements for the evaluation report. Documents/reports 
meeting the following criteria were included in the synthesis: 

Publication date - Given the recent significant increase in the number of children served 
through school readiness initiatives and the relatively recent increase in the number of 
evaluations of such programs, the goal of this synthesis was to examine data from recently 
reported evaluations. Therefore, reports or studies published between January 1997 and April 
2002 were included in the synthesis. 

Location of the program - Only evaluations of school readiness initiatives located in the 
United States were included. 

Nature of the program - The program must be a publicly funded program or intervention, 
which directly targets children from birth to age five. A stated goal of the initiative is to 
enhance children’s readiness for school. In addition, programs must maintain a component of 
classroom-based services. Programs that solely used either a home visitation or parent 
education approach were not included. Neither were drop-in or childcare programs included. 
Evidence of effectiveness - The report must present some type of child outcome data (e.g., 
achievement tests, retention, behavior problems, special education placement) indicating the 
effectiveness of the program. 

Using this set of criteria, two members of the research team carefully reviewed the remaining 
documents to determine which studies were most relevant to the research questions. Study 
abstract or executive summaries were primarily used to determine if a study met the criteria. In 
instances where the abstract did not provide sirfficient information to make a determination, the 
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full report was reviewed. There was 100% agreement between the coders regarding which 
studies should be retained and which ones should be eliminated from the synthesis. 

Analyzing the Literature 

Describing the Evaluations. Each of the remaining documents was coded using a detailed 
coding system designed to capture information about the nature of the school readiness program 
and the methods used to evaluate its effectiveness (see Appendix A). This included information 
about the number of children served, planned program activities/services, staff qualifications, 
etc. In terms of the evaluation of the program, coding focused on the research design, sampling 
procedures, measures/instruments, and outcome data. 

Assessing the Quality of the Research Design. In the early childhood literature, there is 
considerable variability in the quality of research that has been conducted. While experimental 
design with randomized trials represents the standard by which research is measured, early 
childhood research has often relied on less rigorous methods. This has been particularly true with 
respect to four key elements of design: 

Identification of the comparison group 

Initial/follow-up sample sizes 

Attrition 

Measure of program effects (Barnett, 1995) 

A tightly designed study allows the researcher to more confidently conclude that the results are 
due to the actual treatment/intervention rather than some other effects. 

A careful analysis was conducted to assess the quality of the research studies included in this 
synthesis. Twenty of 32 evaluations provided child outcome data of some type and were 
included in this synthesis review. Data range from student achievement scores and report cards 
to information on children’s attitudes toward school and the need for special services. 

The type of research design was the key element in classifying studies at this stage in the 
process. Data on the design of the program reports were recorded on all 20 evaluation studies 
reporting outcome data. Studies were coded into one of four categories: pre-experimental, 
correlational, quasi-experimental, and experimental (as described in Campbell & Stanley, 1963). 
The codings were verified by an independent rater for five of the 20 studies (25%). The raters 
attained 100% agreement on ratings of the design classification. Studies employing either a 
quasi-experimental or an experimental design were deemed to have sufficient internal validity to 
place confidence in their results and were, therefore, reported separately from the pre- 
experimental studies in the synthesis of outcome data. It is recognized that pre-experimental and 
correlational designs, although not as strong in supporting causal conclusions related to 
outcomes, can contribute to the understanding of school readiness programs. Results from 
studies classified as pre-experimental and correlational designs are discussed in the section 
following the presentation of experimental and quasi-experimental studies. 

Synthesizing the Evaluation Results. In order to synthesize the findings from the school 
readiness evaluations, the studies were grouped according to study design: quasi-experimental 
and experimental versus correlational or pre-experimental. From there, the program 
characteristics and evaluation findings were reviewed and patterns identified. It should be noted 
that in conducting a synthesis on the effects of a program, a quantitative meta-analysis is the 
preferable methodology. For the current synthesis, a full-blown meta-analysis was not possible 
due to limitations of the data provided in the reports. Effect sizes were calculated and presented 
for studies whenever possible; however, this was not possible for all the studies. It should also be 
noted that the data available for the synthesis were limited to publicly available study reports. It 
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is recognized that several of the studies are ongoing, and additional data may have been collected 
but not publicly available at the time the synthesis was conducted. Descriptions of the study 
design and the results may, therefore, be less than ideal. The research team made every effort to 
locate and analyze the most complete data available for each report. Readers are cautioned that 
the unit of analysis for this synthesis is the documents referenced, and these may or may not 
provide a complete description of the program and/or the research or evaluation being conducted 
on the program. 



Results 

The synthesis process revealed a number of findings about the character of school readiness 
initiatives and the outcomes associated with those initiatives. The results presented in this section 
will first address the research questions outlined in the introduction. For the two questions 
related to program outcomes, the discussion will focus specifically on the nine studies coded as 
either experimental or quasi-experimental; this will be followed by a discussion of the studies 
categorized as pre-experimental or correlational. 

Research Question 1; What types of school readiness initiatives have been evaluated, and what 
research methodologies have been used to collect the data? 

Description of Programs Reviewed 

Understanding the outcomes of school readiness programs requires understanding the programs 
in which the children were enrolled. Table 1 (Appendix B) offers a brief description of the 
programs that were included in the synthesis. All of the 20 programs reviewed are ongoing large- 
scale public early childhood education programs. Of the 20, 13 are state-funded readiness 
initiatives administered by the state department of education or another state agency such as the 
Office of School Readiness (Georgia) or the Office of Community Development (Washington). 
Four of the reports focus on federal readiness initiatives such as Head Start, Early Head Start, 
and Even Start, and three of the reports focus on local programs (e.g., those supported by federal 
Title I funding). 

Among the programs, there was some variation according to services offered, duration/intensity 
of the program, and program setting. Most of the programs operated on a nine-month 
(corresponding to school year), half-day schedule. With the exception of Early Head Start and 
Even Start, none of the programs enrolled children before the age of three, and most served 
children for one year just prior to kindergarten. 

In addition to classroom-based services, several of the programs offered a range of assistance, 
including health/dental care, home visitation, hearing/vision screenings, and parental support. 

In almost all of the studies examined, children and their families were low income, and the 
children were considered at risk for school failure. Moreover, there was considerable ethnic and 
geographic diversity across the studies reviewed. 

Assessment of Research Design 

The evaluation of programs and interventions designed to improve outcomes for children is an 
important piece of early childhood research. From these evaluations, we hope to determine if a 
program is being properly implemented and whether or not an intervention is having the intended 
effect on those served. However, being able to make a causal connection between a program and 
its outcomes calls for a well-designed evaluation with randomization. Randomization ensures 
that there are no systematic observable or unobservable differences between children assigned to 
the various treatment conditions except for the effects of the program itself. 
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As illustrated in Table 2, the number of program studies using a pre-experimental design versus a 
quasi-experimental or experimental is fairly even. Among the studies utilizing a pre- 
experimental design, most were a one-group, pre-test/post-test design or a one-group, case study 
design. Although data from these studies may begin to suggest some type of impact on children’s 
readiness, it is difficult to draw conclusions that these programs were responsible for the changes 
noted. As a result, studies using a pre-experimental design are presented separately in the 
synthesis of outcome data. 



Table 2: Research Design Used to Collect Outcome Data 



Evaluation 

Study 


Pre- 

Experimental 

Design 


Quasi- 

Experimental 

Design 


Experimental 

Design 


Does Head Start help Hispanic children? (Currie & Thomas, 
1999) 




X 




2000 - 2001 Head Start/ECAP Outcomes Report - State of 
Delaware (Gamel-McConnick, M., Cummings, M., Cornwell, 
J„ & Kersteter, P.G., 2001) 


X 






Building their futures: How E^rly Head Start programs are 
enhancing the lives of infants and toddlers in low-income 
families. (Love, J. Kisker, E., Ross, C., Schochet, P., Brooks- 
Gunn, J., Boiler, K., Paulsell, D., Fuligni, A., & Berlin, L., 2001) 






X 


An investment in children and families: Year 7 longitudinal 
study report. (Northwest Regional Educational Laboratory, 
1999) 




X 




National Evaluation of The Even Start Family Literacy 
Program (Tao, F„ Gamse, B„ & Tarr, H„ 1998) 


X 






Pre-K Longitudinal Study: Findings from the 1999-2000 
School Year (Henry, G., Gordon, C., Mashum, A., & Ponder, 
B., 2001) 


X 






Head Start FACES: Longitudinal findings on program 
performance, third progress report (Zill, N., Resnick, G., 
Mckey, R., Clark, C., Pai-Samant, S., Connell, D., Vaden- 
Kieman, M., O’Brien, R., & D’Elio, M. January 2001) 


X 






Illinois Pre-kindergarten Program for Children At-Risk of 
Academic Failure: FY 2000 Evaluation Report (Illinois State 
Board of Education, 2001) 


X 






Four-Year-Old At-Risk Program: Final Evaluation 
(Martinez, S., 2002) 


X 






Kentucky Preschool Project (Hemmeter, M. L., 2000) 




X 




Educational transitions in early childhood, middle 
childhood, and early adolescence: Head Start vs. public 
school pre-kindergarten graduates (Marcon, 2000) 




X 




Children entering school ready to learn: School readiness 
baseline information. School year 2001-2002 (Maryland State 
Department of Education, 2002) 


X 






Points of light: Third year report of the Michigan School 
Readiness Evaluation (Xiang, Z., Schweinhart, L., Hohmann, 
C., Smith, S., Storer, E., & Oden, S., 2000) 




X 




Pre-kindergarten program product evaluation report, 1999- 
2000, School District of Michigan, Saginaw (Kurecka, P. & 


X 
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best copy 



Evaluation 

Study 


Pre- 

Experimental 

Design 


Quasi- 

Experimental 

Design 


Experimental 

Design 


Claus, R., July 2000) 








School entry assessment project: Report of findings 
(Pfannenstiel, J., 2001) 


X 






Head Start’s impact on school readiness in Ohio: A case 
study of kindergarten students (Cogswell, S., Lochtefeld, S., 
Skaggs, A., & Walker, J., 1998) 




X 




2000 kindergarten survey report: Readiness to learn (Oregon 
Department of Education, 1999) 


X 






Chicago Child-Parent Center Program (Reynolds, A., 
Temple, J., Robertson, D., & Mann, E., 2001) 




X 




A six-county study of the effects of Smart Start childcare on 
kindergarten entry skills (Maxwell, K., Bryant, D., Miller- 
Johnston, S., 1999) 




X 




Kindergartner’s skills in Smart Start counties in 1995: A 
baseline from which to measure change. (Maxwell, K., 
Bryant, D., Keyes, L., & Bernier, K., July 1997) 


X 






Total by Design Type 


11 


8 


1 



Experimental design was used in only one of the studies— the Early Head Start study conducted 
by Mathematica (see Table 3, Appendix B). In this study, children and families were randomly 
assigned to either a treatment group or a control at the time the intake was conducted. 
Consequently, all families were eligible for Early Head Start. This type of comparison group 
(i.e., waitlist comparison) provides a good test of a program’s effectiveness because children and 
their families are both eligible to receive services and motivated to apply for the program. Those 
assigned to the treatment group received the full range of Early Head Start services, including 
child development and education services, healthcare screenings/treatment, family support 
services, and family involvement activities. Those assigned to the control group were not 
allowed to receive Head Start services until their children reached the age of three, although they 
were eligible to receive other services within the community. The researchers also attempted to 
control for non-response bias in their design. Therefore, the experimental design used in this 
study yields the highest degree of confidence in the causal relationship between the early 
childhood program and the reported outcomes. 

Eight of the studies used a quasi-experimental design to gauge the impact of programs on student 
outcomes. In most instances, a non-equivalent control group was used (see Table 3, Appendix 
B). Typically, studies compared program participants with a group of non-program participants 
and attempted to control some of the potential bias by matching the two groups on related 
variables (e.g., race, gender, economic differences). This type of comparison — comparing 
children who received a particular intervention with children who did not— is methodologically 
the best for answering the question of whether a particular intervention had an impact on 
participants. However, as noted in a recent symposium on wide-scale early childhood assessment 
systems (Scott-Little, Brown, and Dufford-Melendez, 2001), school readiness programs are 
finding it increasingly difficult to locate comparison groups with non-served children. As 
services for pre-kindergarten children have expanded, the population of totally unserved children 
has decreased. A large proportion of preschool-age children are being served in some type of 
program, be it a school readiness program or childcare. Therefore, many programs have begun to 
compare children in one program with children in another program. Although not ideal from a 
pure methodological standpoint, it was determined that these studies can contribute to our 
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understanding of the child outcomes associated with participation in school readiness programs, 
and studies with this type of research design have, therefore, been included in this analysis. The 
Smart Start (Maxwell, Bryant, & Miller- Johnston, 1999), Washington, DC Public Schools 
(Marcon, 2000), Chicago Child-Parent Center Program (Reynolds, Temple, Robertson, & Mann, 
2001), Ohio Head Start (Cogswell, Lochtefeld, Skaggs, & Walker, 1998), and Currie & Thomas 
(1999) included comparisons of children participating in multiple types of services, as well as 
children who did not participate in a school readiness intervention in some of the studies. Please 
see the aimotated bibliography that accompanies this report for further details about the 
methodologies utilized in each of the studies. 

Research Question 2: What are the outcomes associated with participation in school readiness 
programs? 

To examine the outcomes and effects of the programs included in this synthesis, data regarding 
measures, outcomes variables, statistical analyses, and effects were coded and summarized. 
Studies using experimental or quasi-experimental designs will be reviewed first, followed by 
studies using other designs. Table 4 (Appendix B) presents an overview of the instruments used 
and the outcomes assessed. 

School Readiness Outcomes Assessed and Instruments Used 

Outcomes were categorized into three major areas: child development, school performance, and 
long-term outcomes. Each area was then subdivided into domains (see Table 4, Appendix B). 

For example, the area of child development includes five domains: children’s overall 
development, language/literacy skills, logic/math skills, child health/physical development, and 
social/emotional development. The area of school performance focuses on academic 
achievement test results, retention, special services (e.g., special education referral, 
speech/language services), attendance, and school adjustment/attitude. Long-term outcomes 
included two domains: dropout rate and arrest record. 

All but two of the programs, Marcon (2000) and Reynolds et al. (2001), conducted assessment in 
the area of child development. Six of the nine programs focused on school performance 
outcomes, and only one program (Reynolds et al., 2001) examined long-term outcomes. Six of 
the programs reviewed assessed child outcomes in two areas — usually child development and 
school performance. A closer examination of the specific domains indicates that measures of 
language/literacy skills were the most commonly used. After language/literacy, children’s social 
development, child health/physical development, and logic/math skills were the domains most 
commonly assessed. 

In terms of instruments used, a wide variety of tests and procedures were employed among the 
1 1 studies. Many of these are well-known, psychometrically valid instruments such as the 
Peabody Picture Vocabulary Test (PPVT), the Woodcock-Johnson, the Bayley Scales of Infant 
Development, and the Social Skills Rating Scale (SSRS). However, others were relatively 
unknown, locally developed instruments with little data regarding their validity or reliability, 
such as Student Behavioral Form (Washington State’s ECEAP). 

Outcomes of School Readiness Programs 

Effects on Child Development 

As noted above, the area of child development focused on outcomes related to children’s overall 
development, language/literacy skills, math/logic, child health/physical development and social 
development. These areas are central to children’s overall development and learning. 
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Before examining the data on child outcomes, it is necessary to consider the strengths and 
limitations of the research design and methodologies used within this group of studies. As noted 
above, some of the studies included less than ideal comparison groups because children in at 
least one of the comparison groups had not been randornly assigned and had experienced some 
type of preschool services. In addition, some of the studies (Marcon, 2000; Cogswell et al., 1998) 
did not take steps to assess whether families who selected one type of program or another (or 
who received no services) were comparable. Attrition was another issue in a few of the studies, 
particularly those that followed children over longer periods of time (Marcon, 2000; Xiang, Z., 
Schweinhart, L., Hohmann, C., Smith, C., Storer, E., & Oden, S., 2000). Typically, there was no 
reported analysis of how subjects who dropped from the program or study differed from those 
who did not. It is acknowledged that our analysis for this review is limited to publicly available 
reports and may, therefore, not present a complete picture of each of the research projects. In 
short, even though these studies have utilized the relatively rigorous designs for examining 
causal relationships between program interventions and child outcomes, we must be cautious in 
interpretations of the results. Given these considerations, we now turn to a description of the 
results reported in the quasi-experimental and experimental studies. 

Language! Literacy Skills. Literacy and language development are key areas in children’s 
ultimate success in school. Many experts believe children’s early experiences with oral language 
development, their interest in books, and efforts to “write” or draw with scribbles begin to build 
a strong foundation for reading success. 

Among the program studies reviewed for this synthesis, measures of children’s language/literacy 
skills focused primarily on children’s knowledge and understanding of print, their ability to 
recognize letters and identify sounds, and their ability to comprehend. Eight of the nine programs 
with experimental or quasi-experimental research designs measured the effects of the program on 
language and literacy skills. Three of those programs found significant effects for program 
participants. For example, small positive effects were noted in the Early Head Start (EHS). This 
study found that, at 24 months, children in EHS programs were using a larger number of words, 
and they were more likely to use grammatically complex phrases and sentences than were 
children in the control group (Love, Kisker, Ross, Schochet, Brooks-Gunn, Boiler, Paulsell, 
Fuligni, & Berlin, 2001). Currie & Thomas (1999) also found benefits for Hispanic children 
attending Head Start. Participation in Head Start had a positive effect on children’s performance 
on the Peabody Picture Vocabulary Test (PPVT). A third study evaluating the impact of a state 
pre-kindergarten program (Michigan School Readiness F*rogram) found significant results in 
language/literacy for kindergarten children (d = .33). 

Math/Logic. In addition to language/literacy, several of the programs included in this 
review examined children’s knowledge and understanding of math. Typically, math/logic skills 
center around one’s ability to understand numbers/ shapes, solve mathematical operations, and 
identify solutions to problems. Of the nine programs reviewed, four included some measure of 
children’s knowledge and skills with numbers. Only one of the programs (Currie & Thomas, 
1999) collecting data on children’s early math skills reported significant findings. According to 
Currie & Thomas (1999) Hispanic children who participated in Head Start scored higher on 
measures of math/logic than their siblings who attended other preschools or who did not attend 
preschool at all. 

Social Relations. Although the focus of most readiness initiatives has been on children’s 
cognitive development and school success, recent reviews (Peth-Pierce, 2000) affirm the 
importance of cluldren’s social/emotional development in determining school success. 

Generally, children’s social skills are conceptualized along two dimensions— prosocial behavior 
and problem behavior (Meece, 1997, as cited in West, Denton, & Germino-Gausken, 2000). 
Prosocial behavior tends to refer to behaviors that facilitate interaction such as sharing, turn- 
taking, and cooperating with others. In contrast, problem behaviors (e.g., fighting, arguing) tend 
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to interfere with good social relationships. Of the nine programs reviewed, six included some 
measure of children’s social competence, and five reported significant differences between 
groups of children. In one of the studies, effects were found for kindergarten children (Michigan 
School Readiness Program). For example, students who had participated in the Michigan School 
Readiness Program scored significantly higher on the social relations subscale than students vyho 
had not participated in the program (d = .30). Looking at slightly younger children, preliminary 
results of the Early Head Start study indicated that, at two years of age, children participating in 
Early Head Start programs display lower levels of aggression as reported by parents. 

Two of the state pre-kindergarten programs found significant results in favor of the comparison 
groups (Ohio Head Start and Washington), while one program found no differences between 
groups on measures of social competence (Smart Start). 

Child Health/ Physical Development. Children’s physical health and well being are 
important precursors for success in school. Children who are healthy, alert, and energetic are 
able to give their full attention to learning in school (National Education Goals Panel, 1997). 
Moreover, how well children are able to perform the typical motor skills/coordination for their 
age may influence how well they do in school. For example, children with poor fine motor skills 
will have difficulty holding a pencil, which will, in turn, affect their ability to print letters and 
words. 

Of the nine program studies reviewed, five included some measure of children’s physical 
health/development. Two of the studies (Smart Start and Michigan School Readiness Program) 
reported significant differences between groups of children with small to moderately sized 
positive effects noted for children who participated in the program (d = .29 and d = .29, 
respectively). Another two programs (Early Head Start and Washington’s ECEAP) found no 
difference between children on measures of physical development/health. 

School Performance 

School outcomes were measured by attendance, rates of grade retention, school adjustment, 
special services referrals/placement, and performance on standardized achievement tests. 

Attendance. Three of the program studies documented school attendance as a measure of 
school performance. Two of the three programs reported no significant difference between the 
program group and comparison group for kindergarten attendance (Ohio Head Start and 
Washington’s ECEAP). The third program (Michigan School Readiness Program) study found 
higher first-grade attendance rates for program attendance, although the effect was very small (d 
= .07). 



Retention. Retention is another key variable in determining children’s success in school. 
Statistically significant effects on grade retention were found in three of the five studies that 
reported collecting relevant data (Currie & Thomas, 1999; Reynolds et al. 2001; Xiang et al., 
2000). A fourth study (Marcon, 2000) found no significant difference between the program 
participants and a comparison group. The fifth study reported collecting retention data; however, 
findings for this information were not described in the document (Washington’s ECEAP). 

School Adjustment. Looking next at adjustment and attitude toward school, one of the 
two program studies that collected data (Washington’s ECEAP) found no significant difference 
between program participants and the comparison group. The second study (Michigan School 
Readiness Program) found that grade 2 teachers rated MSRP students significantly higher in 
interest in schoolwork than children who had not participated in the program (d = .14). 
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special services referral/placement. Children’s placement and referral to special 
education services was another variable for determining the benefits of early intervention. 

Special education services may be measured in several ways. For example, programs may simply 
document whether a child has been referred for special services. Alternatively, programs may 
focus on the number of years children receive special services or the number of special services 
that children participate in. Of the nine programs reviewed, three (Washington’s ECEAP; 
Marcon, 2000; Reynolds et al., 2001) included a measure indexing the receipt of special services. 
For example, Marcon (2000) found that special education placement of Head Start vs. pre-K 
graduates did not differ in either third or fourth grades. In terms of long-term effects, Reynolds et 
al. found that children with extended program participation experienced lower rates of special 
education placement (13.5% vs. 20.7%) at age 18. 

Achievement tests. Finally, only one program used achievement tests to evaluate its 
effectiveness. Such tests tend to focus on academic accomplishments and are a measure of what 
has been taught in school. In a study conducted by Marcon (2000), children enrolled in the public 
pre-K program scored higher on the language portion of the Comprehensive Test of Basic Skills 
(CTBS) than did children enrolled in the local Head Start program. 

Long-Term Outcomes 

Of the studies included in this review, only one, Reynolds et al. (2001), has continued to follow 
children beyond the formal school years. The most recent evaluation of the Chicago Child-Parent 
Center focused on the long-term effects of this program— school dropout, high school 
completion, criminal arrests, grade retention, and special services. The results of this study 
indicated that at age 20, children who had participated in the preschool intervention had higher 
rates of high school completion and lower rates of school dropout. Participation in both 
preschool and school-age intervention were significantly associated with lower rates of retention 
and special education services at age 18. 

Research Question 3: What practices seem to best prepare children to enter school ready to 
learn and succeed? 

The third question driving this synthesis focuses on program effects associated with particular 
program features. That is, are there particular programs or constellations of services that are 
more effective than others? To answer this question, we began by examining results both within 
and across programs. Ideally, one would try to associate particular child effects with specific 
program features. However, none of the experimental and quasi-experimental studies provided 
this type of analysis. Rather, details about the program are provided but are not considered as 
variables for comparisons of child effects. Another factor that limits our ability to draw 
conclusions is the fact that the programs examined did not differ substantially from one another 
on several key program variables, such as the intensity of services. Finally, process features of 
program quality (such as teacher-child interactions) typically were not reported. We know from 
the generd early childhood education literature that process features can have a tremendous 
impact on child outcomes associated with early childhood programs. Therefore, it was difficult 
to answer the third research question — “Which practices best prepare children for success in 
school?”— from the studies that have been conducted. However, some preliminary implications 
from the data or findings are presented in the discussion below in order to shed some light on the 
current state of research within wide-scale publicly funded preschool programs and illuminate 
areas in need of further research. 

The Quantity of the Program 

It has long been held that the more program children experience, the greater the benefit. When 
we look at program dosage, there are three general elements of dosage that must be 
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considered — age at entry, intensity, and follow-through. See Table 1 (Appendix B) for a 
summary of these features from each of the programs included in the review. 

Age at entry. It has been suggested that the earlier an intervention begins, the greater the 
benefit to children and families; however, evidence of this is often confounded by other timing 
variables such as duration and intensity (Bryant & Maxwell, 1996). To some extent, this may 
depend on the program’s goal or expected outcome. That is, if the goal is to improve the parent- 
child relationship, then early entry into a program focusing on parental sensitivity/responsiveness 
may help create a stable and secure attachment relationship. In contrast, a program focusing on 
early literacy and reading skills may not initiate treatment until children are age four. 

Studies included in the synthesis typically began serving children at about the same age. With the 
exception of the Early Head Start program, children entered the program at age three or four and 
exited at age five, just prior to kindergarten. This limited variation in the age of entry meant that 
it was difficult to discern any specific effects associated with serving children at an earlier age. 
Perhaps as Early Head Start follows children into later years, and other studies of programs that 
begin during infancy are conducted, data will emerge to allow an analysis of program effects by 
the age of child entry. 

Intensity. Intensity typically reflects the amount of time that is actually spent in direct - 
contact with the program. That is, how many hours per day, how many days per week, and how 
many weeks per year. Again, many believe that the greater the intensity, the more lasting the 
benefit. Moreover, there is some speculation that there is some minimal level of intensity that 
must be met before program benefits are apparent. 

Looking across the programs included, most served children part day for at least four days per 
week. Only one program (Washington’s ECEAP) allowed children to be served less than four 
days per week (minimum of three days/ten hours per week). At least three of the programs 
(Marcon, 2000; Reynolds et al., 2001; Love et al., 2001) allowed children to be served, at least 
part-day, five (fays per week. To gain a better understanding of the relationship between program 
intensity and child outcomes, it will be necessary to examine programs that serve children full- 
day and also full-year, as well as the part-day, part-year programs examined in studies included 
in this synthesis. 

Duration or follow-through. In this case, follow-through refers to the length of time that a 
child has spent in a school readiness program, plus services provided once the child enters 
school. Among the studies reviewed in this synthesis, the Chicago Child-Parent Center 
(Reynolds et al., 2001) perhaps demonstrates the best example of this program feature. The 
Chicago Child-Parent Center (CPC) is a school-based early intervention progr^ that provides 
comprehensive educational and family support services to disadvantaged families beginning in 
preschool (ages three or four) and extending into early elementary school (second or third grade). 
Evaluations of the CPC have consistently found that children who participate in this program 
score better on measures of school achievement and motivational development. Moreover, 
participation also is associated with higher levels of parent involvement (Reynolds et al., 2000). 
Other evaluations of the CPC (Fuerst & Fuerst, 1993) have found that children who participated 
in the program for at least four years have higher eighth-grade achievement scores and higher 
rates of high school graduation than do children with no participation. 

Program Quality 

Beyond intensity and duration, there are structural factors related to program quality that seem to 
produce better child outcomes. For example, programs that maintain smaller groups of children 
and lower staff-child ratios are associated with better social and cognitive outcomes. Moreover, 
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programs that employ staff with higher levels of training and education tend to be associated 
with more positive cognitive and social outcomes. 

Looking first at child-teacher ratio and group size, research supports the view that small class 
size and low teacher-child ratio contribute to benefits for young children (Frede, 1995). Almost 
all of the studies reviewed for this synthesis maintained some standard for ensuring that children 
were served in small groups with adequate access to caregivers (see Table 1, Appendix B). In 
fact, most of the programs maintained group sizes and ratios that were consistent with those 
recommended by the National Association for the Education of Young Children (NAEYC). 

In terms of teacher qualifications, again the literature supports the importance of highly qualified 
staff who imderstand the importance of child development and have the training needed to apply 
that knowledge as they work with young children. Of the programs reviewed, five (Ohio Head 
Start, Michigan School Readiness Program, Kentucky Preschool Program, Marcon (2000), and 
WasWngton State’s ECEAP) required that teachers have earned a bachelor’s degree and obtained 
a teaching certificate in early childhood education. Another four programs (local or federal Head 
Start programs) required that the lead teacher in each class have earned a Child Development 
Associate (CDA) credential or an associate’s degree (AA). 

Services Offered 

Finally, there is some evidence that successful programs provide children with enhanced services 
such as home visits, family support, health care/screening, and supplemental activities for 
families. Several of the documents reviewed (Love et al., 2001; Cogswell, 1998; Currie & 
Thomas, 1999; Marcon, 2000) were evaluations of Head Start programs (i.e., federally fimded 
and state expansions of Head Start services). Unlike many other programs that provide services 
to children. Head Start has a broader mission that requires programs to provide a comprehensive 
set of services, including educational, health (i.e., medical, dental, and mental health), 
nutritional, and family support services. However, other programs offered supplemental services 
as well. For example, the Michigan School Readiness Program provided home visits by staff, 
parent education classes, and scheduled family events. Washington’s ECEAP program provides 
f^amilies with case management services, access to support groups, and home visits by staff. 
Finally, the Chicago Child-Parent Center provided families with comprehensive health, 
education, and family services (Reynolds et al., 2001). It seems possible that these more 
comprehensive programs might yield greater benefits in terms of child outcomes. However, data 
from the studies included in this review did not address this issue directly. 

Methods and Findings from Programs Using Non-Experimental Design 

As previously mentioned, there were 1 1 program reports/articles that did not use a comparison 
group (see Tables 5 and 6, Appendix B). Although our ability to draw causal conclusions is 
limited, these pre-experimental studies can contribute to our understanding of school readiness 
interventions. Many of these studies collected more data on the implementation of the programs 
and some of the quality variables that were noted as. lacking in the experimental and quasi- 
experimental studies described above. Researchers and evaluators conducting these studies have 
most likely selected the particular design for a specific reason or reasons. For example, the pre- 
experimental designs usually are more practical for programs. In addition, the purpose of the 
research may lend itself to a pre-experimental design that focuses more on program 
implementation than comparison of child effects. Finally, the pre-experimental studies may be 
exploratory in nature, providing critical information to inform future research questions for 
experimental or quasi-experimental research. In short, these studies can make a valuable 
contribution to the literature. What follows is a review of the 1 1 studies that utilized pre- 
experimental or correlational research designs. The methods and findings from these studies are 
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summarized. Readers can find additional details about these studies in the Annotated 
Bibliography that accompanies this report. 

In most instances, the data collection methods in these studies were primarily a one-group, pre- 
test/post-test design. Most commonly, a pretest was administered to a group of children who had 
participated in the pre-kindergarten program, and a post-test was administered to see what 
changes had taken place during the time they were in enrolled in the program. A large number of 
these reports relied on checklists, teacher ratings, or teacher survey results to estimate program 
impacts. These ratings usually focused on three domains of development: cognitive, 
social/emotional, and physical development. For example, an evaluation study of North 
Carolina’s Smart Start initiative (Maxwell, Bryant, Keyes, & Bernier, 1997) asked 858 
kindergarten teachers to rate incoming kindergarten students on their pre-academic and social 
skills. According to the teachers surveyed, approximately 18% of kindergartners were not ready 
at the begi nni ng of the year to participate successfully in school. A survey also was conducted 
among kindergarten teachers in Oregon (Oregon Department of Education, 1999). Using the 
Kindergarten Readiness Survey, teachers were asked to rate children’s skills as they entered 
school. More than 900 kindergarten teachers rated children’s development along various 
dimensions of readiness including language usage, approach to learning, cognition/general 
knowledge, social/emotional development, and motor development. Results from this survey 
indicated that approximately 65% of children met all of the readiness dimensions. When 
researchers examined children’s readiness in the context of prior care experiences, they found 
that 88.6% of children who attended preschool were deemed “ready” on at least five of the 
readiness dimensions. This is in contrast to 69.7% of children who attended Head Start and 
52.6% of children who attended the Early Childhood Special Education program. 

Similar percentages of children’s readiness were found in a report of the Missouri School Entry 
Assessment Project (Pfaimenstiel, 2001). A random sample of Missouri schools (10%) was 
selected for participation in an evaluation of children’s readiness. From these schools, all 
kindergarten teachers were designated as participants, as were the children in their classrooms. 
Using the School Entry Profile, teachers were asked to rate children’s preparedness for 
kindergarten. Overall, approximately 29% children were rated as “above average” in their 
preparedness for school, 21% were rated “below average,” and 50% were rated as “average.” 
Differences also were noted according to prior preschool experiences. Children who were in 
home-based care during the pre-kindergarten years performed lowest among kindergartners 
assessed, scoring lower on all domains. Children who attended preschool (e.g.. Early Head Start, 
Head Start, and public preschools) and children who attended childcare programs scored average 
or slightly above average on all scales. The highest-performing children were those who 
participated in Parents as Teachers, plus some form of preschool. These children scored 
significantly higher on all domains of the School Entry Profile. 

Two of the programs reviewed relied on elements of the Work Sampling System (Meisels, 
Jablon, Marsden, Dichtelmiller, & Dorfman, 1994) to measure children’s readiness for school. 
The Work Sampling System is an assessment tool that was originally designed to improve 
teaching and instructional strategies; however, some states have adapted the instrument for use in 
determining what skills and abilities children bring with them at kindergarten entry. In 2002, the 
Maryland State Department of Education collected baseline information on incoming 
kindergarten students (Maryland State Department of Education, 2002). Results indicated that 
almost half (49%) of the students were rated “full readiness” by their teachers. Forty-four percent 
were rated as “approaching readiness,” and seven percent were rated at the “developing 
readiness” level. In terms of children’s previous care experiences, children who had participated 
in the “non-public nursery” program (i.e., preschool programs with an education focus) were 
rated “more ready” than children who participated in other forms of care. In contrast, children 
who were in family childcare or informal care with a relative were rated “less ready” than 
children in other forms of care. 
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Evidence from another school readiness initiative also seems to suggest that early childhood 
programs can have a positive impact on students’ entry into kindergarten; however, it also raises 
questions about the sustainability of those effects. Evaluation of Kansas’ Four- Year-Old At-Risk 
Program found that preschool experience was related to children’s proficiency at the beginning 
of the kindergarten year in all domains of the Work Sampling System Checklist (Martinez, 

2002). Students with preschool experience were still at a slight advantage in the spring of the 
kindergarten year; however, by the first grade, there were no statistically significant differences 
in proficiency ratings between children with and without preschool experience. 

Four of the studies coded as pre-experimental (Head Start/FACES study. Even Start Study, 
Illinois, Delaware, and Saginaw, Michigan) included direct measures of student performance. 
Looking first at the Head Start Family and Child Experiences Survey (Zill, Resnick, McKey, 
Clark, Pai-Samant, Connell, Vaden-Kieman, O’Brien, & D’Elio, 2001), this study involved a 
nationally stratified random sample of 3200 children and their families. The FACES child 
assessment consisted of a series of tasks designed to assess children’s skills as they enter 
kindergarten. Direct assessments of children’s cognitive development included measures such as 
the PPVT-in (receptive language skills), the Story and Print Concepts, the Phonemic Analysis 
Task from the Test of Language Development III (phonemic awareness), and subtests of the 
Woodcock-Johnson (letter-word identification, problem-solving, and dictation). In addition, - 
information was gathered from parents and teachers to assess children’s social/emotional 
development. Results found that children in Head Start showed significant gains in vocabulary, 
writing skills, and social skills. However, little or no change was noted in other areas of 
development, such as letter recognition and problem behaviors. 

The evaluation of another federal program also found positive gains among the children served. 
Results from the national evaluation of the Even Start Family Literacy program (Tao, F., Gamse, 
B., & Tarr, H., 1998) indicate that participation in an early literacy project may promote 
educational gains for young children. Focusing on a sub sample (10%) of children who 
participated in the universe study, researchers used a statistical model to predict children’s 
growth on the Preschool Language Scale (PLS-3) and the Pre-school Inventory (PSI). They 
concluded that children who remain in Even Start for longer periods of time will grow at a faster 
rate than those who do not remain in Even Start. 

To evaluate early childhood programs in Delaware, researchers randomly selected children and 
families from local Head Start programs, as well as the state-funded Early Childhood Assistance 
Program (Gamel-McCormick, Cummings, Cornwell, & Kersteter, 2001). A total of 495 children 
were assessed in September and again in May with an approved instrument selected by the 
program. Standard scores were established to compare children’s achievement in each of the 
following domains: cognitive, communication, physical, and social/emotional. Results indicate 
that children showed significant skill development in all four areas of development. On average, 
children demonstrated a 27.7% increase in cognitive skills, a 33.9% increase in communication 
skills, a 20.8% increase in physical skills, and a 26.1% increase in social/emotional skills. 

In the only evaluation of a local program, researchers in Saginaw, Michigan, assessed more than 
300 children who participated in the Saginaw Pre-kindergarten Program (Kurecka & Claus, 
2000). Using the fte-Kindergarten Saginaw Objective Reference Test (PK-SORT), researchers 
examined student achievement on a posttest-only basis. The PK-SORT includes 31 items dealing 
with both psychomotor and cognitive program objectives. It is administered to students on a one- 
to-one basis. Findings indicated that the majority of students attained all of the cognitive 
objectives and three of the four psychomotor objectives. The area of lowest attainment was 
expressive language/semantics (83.8%), while the area of expressive language/labeling 
demonstrated the highest level of attainment (98.7%). 
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Two of the programs reviewed have continued to follow students over time and included more 
long-term measures of student success: student retention (Georgia) and student achievement 
(Illinois). The Georgia study represents an evaluation of the Georgia Pre-kindergarten Program 
that was established in 1993 (Henry, Gordon, Mashum, & Ponder, 2001). The evaluation, which 
was initiated in 1996, was designed to follow a probability sample of pre-kindergarten children 
into their elementary school years. Results indicate that, at the end of their second-grade year, 
82% were deemed ready for the third grade, according to their second-grade teachers. More than 
half (56.1%) of the students were judged to be “extraordinarily good” to “good” in their 
preparation for the third grade. Students gained the most in math, science, and language arts. 
Although little change was noted in social behaviors, students’ attitude toward learning and 
independence increased significantly during the year. The study also included a look at the 
relationship between teacher characteristics and child outcomes. Overall, students of teachers 
who believed in and practiced child-centered methods of instruction out-performed others during 
pre-K and kindergarten. Also, student outcomes varied according to the credentials and 
experience level of the teachers. 

With regard to the Illinois Pre-kindergarten Program, results seem to suggest that early childhood 
programs may also have a positive effect on student grades — particularly for those students who 
may have been considered at risk for school failure (Illinois State Board of Education, 2001). 
Focusing on a sample of children who had previously participated in the program, researchers 
found that approximately 55% of third- grade students who participated in the program “met” or 
“exceeded” the standards in writing, while 75% “met” or “exceeded” the standards in 
mathematics (results based on the Illinois Standards Achievement Test). In eighth grade, 41% of 
the students “met” or “exceeded” the standards in mathematics, while percentages in writing and 
science were 64% and 72%, respectively. 

This review of the pre-experimental studies clearly illustrates that researchers use a variety of 
methods to assess the effects of a program on children’s well-being and development. Some 
studies selected teachers from a representative sample of schools and asked them to rate 
children’s readiness on various domains. Others focused on a sample of children attending a set 
of classrooms or programs and selected them for observation or assessment. While there are 
obvious shortcomings to the pre-experimental design, such studies tend to be more practical as 
compared to the experimental design, which is often more difficult and expensive to implement. 
However, if a researcher chooses to employ a pre-experimental design, he/she must take 
additional care to test for and use control variables to minimize the differences between children 
in various groups. 



Discussion and Recommendations 

The recent increase in the amount of funding available for pre-kindergarten services indicates 
that many states, agencies, and local communities are making a commitment to early childhood 
care and education and to evaluation of the programs. A previous study published by Gilliam & 
Zigler (2001) found 13 state evaluations of child outcomes. Just five years later, this review 
yielded an additional eight evaluations of school readiness initiatives, as well as updated reports 
from seven studies included in Gilliam & Zigler (2001). In addition, there were numerous other 
local and federal evaluations. At the same time that programs and agencies are expanding their 
commitment to early childhood services, there is a strong push to demonstrate the value and 
effectiveness of these services. 

Research Findings 

Results from this synthesis indicate that wide-scale school readiness interventions can have 
moderate effects on child outcomes. Perhaps the strongest evidence comes from results 
associated with data collected on children’s social -emotional development, with the largest 
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number of studies reporting children in school readiness programs having more positive social 
development than children in comparison groups. Positive results were also reported for 
children’s language/literacy, mathematical thinking, and physical/health development and for 
student outcomes such as better attendance or fewer referrals for special services once the 
children entered school. Limitations in the research designs and the data reported in the 
documents examined in this synthesis, however, warrant caution when interpreting results. In 
general, most of the results can be characterized as small to moderate, and in each of the child 
outcomes examined, some studies reported positive results while others reported no significant 
differences. 

Relative to particular features of program quality, it was difficult to discern whether specific 
program features are associated with positive participant outcomes. Studies that used 
experimental and quasi-experimental designs typically did not examine relationships between 
specific program features and child outcomes. Some of the most informative data relative to 
features of program quality come, perhaps, from the pre-experimental studies. For example, the 
evaluation study of Georgia’s Pre-kindergarten Program noted significant differences in teacher 
effectiveness that related, not only to teacher training, but also to the number of years of 
experience (Henry et al., 2001). Specifically, the effectiveness of pre-kindergarten teachers who 
held a CDA declined as their years of experience increased. A similar decline was noted among 
teachers who had obtained a college degree, but who had not obtained certification. Interestingly, 
certified teachers were rated as less effective early in their careers; however, their effectiveness 
increased or remained constant as they gained experience. The authors speculate that perhaps 
certified teachers receive more effective training that allows them to put their knowledge to 
better use as they acquire more experience. Clearly, there is still much to learn about the 
relationship between features of program quality and outcomes for children. 

Implications for Program Evaluation 

Accountability for public spending is a factor driving many of the program evaluations that have 
been put into place. However, properly evaluating the effectiveness of a program or initiative can 
be challenging, and there is not yet a standard approach to conducting the evaluation. 

Among the reports reviewed, there was considerable variation in the design of the programs, the 
overall objectives defined, and the services offered. Evaluation of these divergent programs is 
further complicated by the variation in the need for data and purposes for evaluations. Different 
stakeholders— community representatives, practitioners, policymakers, researchers, and 
families — may have different levels of need with regard to ev^uation results. What satisfies one 
stakeholder may be inadequate for another (National Governor’s Association, 2000). 

In general, most of the programs reviewed demonstrated that they are beneficial to children, 
particularly low-income children; however, it is clear that many of these efforts must be 
accompanied by more carefully designed studies to ensure that we have a complete 
understanding of how these programs work and what their benefits are. For example, only one of 
the studies reviewed (Love et al., 2001)— a study of the Early Head Start program— used a 
randomly assigned control group. The remainder used less rigorous comparison groups such as 
matched program-eligible non-attendees or program eligible non-attendees who did not attend 
any other program. Moreover, some evaluations, such as Ohio Head Start (Cogswell et al., 1998) 
and Currie & Thomas (1999), used multiple comparison groups. The type of comparison group 
used can have a tremendous effect on the validity of the findings and how the findings are 
interpreted (Gilliam & Zigler, 2001). Gilliam, Ripple, Zigler, and Leiter (as cited by Gilliam & 
Zigler, 2001) suggest that it may be unfair to test the effectiveness of a program by comparing 
outcomes of children who have not participated in a similar program. This may bias the study 
toward positive results. In contrast, however, Schweinhart (1999) maintains that it is almost 
impossible to compare the effectiveness of Head Start, public pre-kindergarten programs, and 
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preschool childcare programs because they serve very different populations. Head Start serves 
primarily poverty-level families, and while income may be a consideration for public pre- 
kindergarten programs, it is usually one of several factors determining en^ into the program. In 
terms of preschool childcare, enrollment into this type of program often hinges on the families’ 
need for care so parents can work outside the home. Not only does this increase family income, 
but it also makes it extremely difficult to randomly assign children who need care to an unserved 
control group. Although it may not always be possible (or even desirable taking ethical concerns 
into consideration) to conduct randomized trials, these types of studies are the most effective for 
determining if the difference in outcomes is due to program intervention (Karoly et al., 1998). 

Assuming one can establish an appropriate sample, it is then important to continue to follow 
those subjects across time in order to capture both long-term and short-term effects. As seen 
among studies of model programs (Campbell & Ramey, 1994; Schweinhart et al., 1993), there is 
evidence suggesting that early childhood programs may help buffer children against later 
problems such as delinquency or antisocial behaviors, as well as help promote later achievement 
(i.e., higher rate of graduation and lower dropout rate) (Yoshikawa, 1995; Reynolds et al., 2001). 
Unfortunately, following subjects over time often results in losses due to attrition. Attrition rates 
for studies included in this synthesis ranged from approximately 17% to more than 50%. Without 
an analysis of how the participants who dropped from the study compare with those who 
continued, our ability to draw conclusions from the research is limited. 

In addition to designing more rigorous evaluations, researchers and program evaluators must also 
give attention to the methods and instruments used to monitor change in yoimg children. The 
recent push toward accountability places further burden on both the assessment instrument and 
those responsible for collecting the data to accurately and fairly capture the effects of a program. 
As noted earlier, a potential problem with several of the studies was the use of tests with little 
known validity or reliability. However, this also calls into question the role of teachers and 
educators in the data collection process. In instances where teacher judgment of student learning 
is a key element of performance assessment, some researchers (Hoge & Coladario, 1989) have 
expressed concerns about the validity and reliability of teacher assessment, particularly in high- 
stakes situations. However, others (Meisels, Bickel, Nicholson, Xue, Atkins-Bumett, 2001; 
Schweinhart, in press) maintain that teachers can collect data that is both reliable and valid. 

These issues highlight the need to collect data in a consistent and standardized way that permits 
comparisons across classrooms or programs. 

Another issue affecting the selection and use of assessment measures is the nature of children’s 
development. As asserted by the National Education Goals Panel (1995) and reaffirmed in a 
recent report issued by the National Research Council and Institute of Medicine (2000), 
children’s readiness is multidimensional. Moreover, young children’s development is often 
sporadic and uneven. Consequently, researchers and evaluators must be careful to select 
instruments that capture children’s full range of developments, as well as instruments that are 
reliable and valid. A recent article by Zaslow & Halle (in press) concludes that work is needed to 
strengthen assessments of certain dimensions of school readiness. The authors specifically 
mention the need to create better measures of children’s social/emotional development. Current 
measures, particularly those measuring positive aspects of young children’s social/emotional 
development, are limited by psychometric problems. 

Beyond research design and instrumentation issues, researchers and program evaluators must 
also look at program quality and the dynamics that occur within the classroom. One cannot fully 
understand child effects without first considering the quality of the program. If a program fails to 
demonstrate positive results, absent a measure of quality, it is difficult to determine whether the 
results are due to a poorly designed program model or a poorly implemented program model 
(Gilliam & Zigler, 2001). With the exception of the Head Sta^FACES study (Love et al., 2001) 
and the Michigan School Readiness Evaluation (Xiang et al., 2000), none of the programs used 
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quality as a means to better understanding their findings. Other program evaluations such as 
Georgia’s (Henry et al., 2001) surveyed an element of quality, namely teacher beliefs/attitudes; 
however, there was not a separate measure of program quality. One benefit of examining 
program quality and child outcomes would be to determine if there are specific program aspects 
that should be the focus of more intense technical assistance or in-service training. 

Although this synthesis just begins to touch on some of these issues, the programs reviewed help 
provide some insight. For example, the evaluation of the Chicago Child-Parent Center (Reynolds 
et al., 2001) suggests that the range of services and the extent to which these services are able to 
follow and support children as they enter school may be an important program factor. Moreover, 
these studies also seem to suggest that while it may be useful to intervene before the age of three, 
preschool and school-age interventions can also be effective. 

There continue to be a number of unanswered questions about program design, and clearly, more 
research is needed. For example, at what point is it best to initiate services— infancy or 
preschool? Does the level of intensity make a difference, and if so, do the same levels benefit all 
children? What types of services or combination of services yield the best outcomes? These are 
just a few of the questions that the early childhood community struggles to answer, and 
evaluations are an important means for understanding how investments in early childhood 
initiatives will pay off. 
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Appendix A 

ELO Research Synthesis 2002 
Data Coding Sheet 

REPORT IDENTIFICATION 

1. Synthesis Strand: Early Childhood School-age 

2. Report Code #: 3. Publication Date: 

4. Source of Report (Select one) 

a. Journal article (refereed) 

b. Journal article (non-refereed) 

c. Conference presentation 

d. Thesis or dissertation 

5. Peer Review Process 

Peer reviewed Not peer reviewed Unknown 

H: Hi* ***************************************************************** ******* 

PROGRAM OR INTERVENTION INFORMATION 

Type of Program (Select all that apply): 

School Readiness: 

Head Start 

Pre-K or other school readiness program 

Even Start/Family literacy 

Other: 

Transition: 

Transition to kindergarten 

Transition between grade levels 

Transition to/from after-school 

Transition beyond high school 

Other: 

School-age: 

After-school 

2T‘ Century Community Learning Center 

Instructional Support Programs (Tutoring, Voyager, homework help, etc.) 

Extended School Time 

Extended day Extended Year Summer School Year-round 

Learning Centers (such as Sylvan) 

Technology assisted learning 

Other: 



e. Organization or agency report (published) 

f. Organization or agency report (internal) 

g. Personal communication 




1 



29 



Characteristics of Program/Intervention Participants: 



Student Characteristics: 

Grade 

a. Preschool children 

b. Elementary 

c. Middle school 

d. High school 

e. Parents 

f. Other 

Gender (indicate the percentage) 

a. Female 

b. Male 

c. Not specified 

Special population (specify): 

Setting (location of program) 

a. School 

b. Community-based 

Faith-based 

Non-faith-based 

c. Mixed 

d. Not specified 

Name of City(ies): 

Name of State: 



Race (indicate the percentage) 

a. White 

b. African-American 

c. Hispanic 

d. Other (specify): 

e. Not specified 

Economic level (indicate the majority) 

a. Poverty 

b. Non-poverty 

c. Not specified 

d. Mixed 



Setting 

a. Rural 

b. Urban 

c. Suburban 



Profile of Program/intervention: 

Number of children served/enrolled/involved (describe whether this is planned, 

observed, average, etc.) 

Number of hours of operation 

Duration of program (number of years in operation) 

Source of funding 

a. Public 

b. Private 

c. Fee-based/for profit 

d. Mixed 
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He************************************************************************ 

EVALUATION STUDIES INFORMATION 



Type of Evaluation: 

a. Formative 

b. Implementation 

c. Summative 

a. Internal Source of funding for evaluation: 

b. External 



Research Design 



Design Type: 

fte-experimental 

One group or case study 

One-group pretest-posttest design 

Survey 

Other: 

Correlational 

Quasi-experimental 

Nonequivalent control group, post-test only 

Matching No matching 

Matched on: 

Nonequivalent control group, pre-test/post-test 

Matching No matching 

Matched on: 

Time series 

Cohort study 

Other: 

Experimental (with random assignment) 

Post-test only control group 

Pre-test post-test control group 

Other 



Sampling: 

Random 

Judgment or convenience 

Sampling procedure unknown 

If random assignment used, students randomly assigned 

Qasses randomly assigned 

Schools randomly assigned 

Other: 



Sample 

Total N: 

Sample Size: 
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Treatment; 

Control; 

Attrition or Response Rate; 

In treatment group; 

In control group; 

Excessive? Y es No 

Gender; % Female 

Mean Age; SD Age; 

Multiple grades? Y es No 

Race; Percentage minority Not specified 

Treatment/Intervention 

Independent variable; describe how the independent variable was administered; 
Description of intervention; 



Characteristics of intervention; 

Adult;child ratio; 

Education of adults (staff or volunteers); 

High school 

Some college 

College grads 

College plus 

Specialized training for adults in program; 

No 

Yes 

Please describe; 

Length of intervention per week (hours); 

Duration of program per year (months); 

More than one year for average participants? No Yes 

Number of years; 

Program quality/model implementation validated? Yes No 

If yes, how? 

Families involved in program? Yes No 

If yes, describe how; 



Any variation in treatments among study groups; 



Control group; 

No treatment 
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Placebo 

Treatment delayed 

Potential problems with control group 

Control group from different time period 

Control group from a different grade 

Multiple grades combined 

Control group pre-existing or convenient 

Other: 



Theory 



Specified: 
Inferred: _ 
None 



Measurement of Dependent Variables 

Type of Dependent Variable Measure (Check All That Apply): 

Written measure completed by subject Informal 

Verbal Standardized 

Behavioral 

Interpersonal judgment 

Program/School records 

Reliability established: Yes No 

Type of reliability measures: 



Same data collected for treatment and control group? Yes No 

Results (Include effect sizes if available) 



Outcome measure 


Positive 

effect 


No 

effect 


Neg. 

effect 


Comments (include whether 
results statistically 
significant) 


a. 










b. 










c. 










d. 










e. 










f. n 






















Other results: specify 
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Statistical analysis 



Pre-test means: Y es 


No 


Post-test means: Y es 


No 


Pre-test standard deviations: 


Yes No 


Post-test standard deviations: 


Yes No 


Significance tests: Yes 


No 


Tvpe of inference test: F 


t 



Does the report include program/intervention recommendations? 

Yes No 

Does the report include recommendations for further evaluations/research? 
Yes No 



******************************************************************* * 

Coder: KDM GB EM CSL Other: 

Date: 
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Schulman, K., Blank, H., & Ewen, D. (1999). Seeds of Success. Washington, DC: Children’s Defense Fund. 
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Table 4: Instruments Used and Outcomes Assessed Among Quasi-Experimental and Experimental Designs 
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Table 5 : Program Characteristics of Pre-Experimental Evaluations 
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Additional Sources: 

Schulman, K., Blank, H., & Ewen, D. (1999). Seeds of Success, Washington, DC: Children’s Defense Fund. 



Table 6: Design Characteristics of Pre-Experimental Evaluations 



B 

© 

H 

» 

O 







CQ ^ 
o £ 

*o o 
•tj s .t: 

•g -S g 

c 3 u 

c o ^ 

w c »c 
OA C O 

ii-S-g 
o S S 

M 42 U 

■C ^ ^ 

«7 § Eo . 

i|w ^ 

l-g s 2 

^ rt C ^ 

• C tL CJ 

= 4| I 
^ll-i 
0 s ii 



i « .. g 
« « s s 
« 'S ^ 

“ fli 

T3 2i 'C S 
e « X ^ 

2 Ic-S 

« ^ g ^ 

i^li 

^•§2° 

o id: 



U S 

ra- 
if I 

> C y 

1 u (/] 

L|^ 



ua2 



€ 3 



04 



'2, ^ 
^ d 

>%.S 



isa = 

.a S-&' u 

S-P I * 

CL C y 52 

- 04 g “ 

o ^ w ” 



o J J 



3 



2 



o o 

tl ® 

O T3 

•C'6 



2 S 

2 ;§ 

•a ^ 

a -fi 



*0 0) 
w ,D 

© 

Sc 

|l 

tg 




W g 8 J3 

^ U tn y 



P ^ 

•S <i> 
•6 g 
W . h 
u 'd 

^ « 

^1 

II 

|1 

*s y 



43 

,o > 
2 -C 

O. O4 



m o 

VO 00 



Si 



s 



3 _ s ■= 

uU 

lili 

’•a "p y S 

aa go 

u " S 
6 ™ “ "S 
6 °aia 
s s l-g 

g g -g 

“I 5 I 

S « « o 

I's^l 

g ^ ^ ^ 

y sq r^ vq 
^ 00 On rj 
Oh 00 NO 



» 

v> 



s 

(M 

o 

& 

►> 

H 



m 2 
CQ y 

E to 

C DA 

y.S 

‘•3 9- 

& g 

MOO 

i 



-a 



OA 



o 
g M 

ll 

8l^ 

J 3 J 3 I 
^ ^ 00 
Si Sitr^ 






le 




a 



g3 

•O y 

26 




O w 



i i 2 II 

j y o y y 
H oaO 0^ ^ 



2 OA O **3 

w -S ^ o 
c2 § fl o § 

g|ai>ii 

OA n ,•'•0 Tt a 
b4 o y y cQ 
y OA o O 4 . 4 ^ 

T3 « W OA « -tt *s 
C CO O ^ G O 
• ^ 43 G y y V y 

c 2 o o 
^ *y *y 



i ^ 

oSi ?• 






O CO 



G I g ^ 



CA 

© 

a 

e 

CQ 

CA 



S8 



a; 

OA-^ 

•d y 
G •© 

2 a 

O 43 
ON a 



■S-a 



« s 

y & 

i’6 




•a y 
V g“ ^ & 
«'| J5-d 

ol ^g 



I s 

G p 
X ti? 

o 

a» 
« g 

y OA 
O4 y 
G *d 

a p 

CO 2 



1 CO 

<^1 
S § 

"I 

o T! 

04 . 

: y 

3 a 

H SQ 



“S 

^ s 

O 



On 

sSi- 

P ^ CN 
cS y 

es22 

y G 

•g 



3,a-d 
uo p y 

>1 2 

2 y ^ Qo 

3 ” a c 

i ^ rt y 

S^.'c a 



> 

O 

Z 

•S 



Is 

y o 
H 



'C y 
y y 

04 « O 

« s S' 

5 2 I 

d:3i 



^J- 

a *-6 
S 2^ 

g M y CO 

'*« .S' 



a;_ 

X o 



if-€ 

■' 2 S 



■G y 

a -a 



^ 5 
13 

4S 

U 

CO 



O a 
"O .5 

& § 

s-g 

IT 

cjs 

»a "n M 

tSs 

OQ M O 



Z 



g 2 4^ 

s>r gS 
« 1 2§ 
If ^1 

" 0^1 

^ a a S 

§ w 

iisj 

oOCS 



u 

1 1 
c h 
^ flu 
CA ^ 



•S = ^ 

i||5 

CO'S* 2J 

s y G 

2 0.2 w 

|“l| 

|i«l 

y ^ 43 

^iil 



«a 

g 

>1 



PQf 

I' 



I - 

■g.2 

a- 2 



^ A 



U w V 



O c3 
CA y 
S 

w ^ "O 

*H a 

a^'s 

S Q e 

III 

^ Ih y 

1 SL a 
i^Q 




^ ^E§T COPY AVAHIABIUS 



B 

o 

u 

6 




« M g >* y 

y o o o 

U U S CO 



u § '■^ 



43 " 

n « 

S s 

T 3 C 

i-g 

fc-'a •“ 

g W ) 

I 8 = 

g a a 

y fc- 

§ -<3 

’§ 

S § C 

c 3 s ■;: 

OO V ^ 

rS & W 

S O ^ 

a 0.0 

ig I 

-S § 



g 

s . 



1 I 



S - c 3 

C oo 
S to 

I > ^ 

2 




^ 00 g 

r^vq I 00 

.s ' 3 ■> 

•3 j 3 oo g 

2 I S-g 





V 

s I 



I 

« 

V 



E? 8 



^ °| § 
p ^ CO t > CO 



•o 

c T 3 

3 S 



u£wS 



^ oo 
G . S 

Ii2 

M > u 

‘S « 
§2 8 



K y-^ y y 

fC oc : a < H 



X 

0» 

a 

S 

69 

CO 




T3 

.2 

*5i) 

'C 



y 

I§ 



8 

<N 

fO 



•S' 

a 

CO 

•c3 

c 

a 



43 

O 

U 



2*? « 

^ S:2 . 

« o y S 

sS3 I 

wi CS i OJJ 

S ^ o 

Sgg & 

y c o c 
2 

2 " y 

Stll 



1 .ir 



(2 B . 



•a 



Pi -.S 

1 3 

■s 



n 

S « 

o O 
^ h ^ 

|0 ii 






Idt 

>S i 

il S - 

» 'S a 

I X K 



-if o 

■ M 

.23 « 
o ^ 




References 



* Denotes document reviewed for the synthesis 



* Barker, N. C. (1998). Can specialized after-school programs impact delinquent behavior 

among African-American youth? Child welfare and juvenile justice. Proceedings of the 
Tenth Aimual Research Conference, A System of Care for Children’s Mental Health: 
Expanding the Research Base, Tampa, FL, 2-7. 

Barnett, W. S. (1995). Long-term effects of early childhood programs on cognitive and school 
outcomes . The Future of Children, 5(3), 25-50. 

Beck, E. L. (1999). Prevention and intervention programming: Lessons learned from an after- 
school program. The Urban Review, 31{l), 107—4. 

Beckett, M., Hawken A., Jacknowitz A. (2001). Accountability for after-school care: Devising 
standards and measuring adherence to them. Rand Corporation. Retrieved from 

www.rand.org/pubIications/MR/MR141 1/. 

Beuhring, T., Blum, R. W., & Rinehart, P. M. (2000). Protecting teens: Beyond race, income, 
and family structure. Minneapolis, MN: Center for Adolescent Health. 

* Bissell, J., Ashurst, J., & Jones, P. (2001). Los Angeles Unified School District After School 

Learning and Safe Neighborhoods Partnerships Program: An evaluation of four agencies 
and supplemental report. Irvine, CA: University of California, Irvine Department of 
Education and Research Support Services. 

* Bissell, J., Dugan, C., Ford-Johnson, A. & Jones, P. (2002). Evaluation of the YS-CARE After 

School Program for California Work Opportunity and Responsibility to Kids 
(CalWORKS). Irvine, CA: Department of Education, University of Calif omia-Irvine. 

* Bissell, J. S., & Malloy, J. (2002). Evaluation of California’s After School Learning and Safe 

Neighborhoods Partnerships Program: 1999-2001 . Preliminary report. Department of 
Education, University of Califomia-Irvine. 

Boruch, R. F. (1997). Randomized experiments for planning and evaluation: A practical guide. 
Thousand Oaks, CA: Sage Publications. 

Boruch, R. F. ( 1998). Randomized controlled experiments for evaluation and planning. 

Handbook of applied social research methods. Thousand Oaks,.CA: Sage Publications. 

Brown, P. (1995). The role of the evaluator in comprehensive community initiatives. In J. P. 
Connell, A. C. Kubisch, L. B. Schorr, & C .H. Weiss (Eds.), New approaches to 
evaluating community initiatives: Concepts, methods, and contexts (pp. 201-225). 
Washington, DC: Aspen Institute. 

Bryant, D. M,. & Maxwell, K. (1997). The effectiveness of early intervention for disadvantaged 
children. In M. J. Guralnick (Ed.), The effectiveness of early interventions (pp. 23-46). 
Baltimore, MD: Brookes. 

Campbell, D., & Stanley, J. (1963). Experimental and quasi- experimental designs for research. 
Boston: Houghton Mifflin. 



ERIC 




Campbell, D. T., & Stanley, J. C. (1966). Experimental and quasi-experimental designs for 
research. Chicago: Rand McNally. 

Campbell, F. A., & Ramey, C. T. (1994). Effects of early intervention on intellectual and 

academic achievement: A follow-up study of children from low-income families. Child 
Development, 65, 684—698. 

Campbell, R. A., & Ramey, C. T. (1995). Cognitive and school outcomes for high risk African- 
American students in middle adolescence: Positive effects of early intervention. 

American Educational Research Journal, 32, 743-772. 

Catalano, R. F., Berglund, M. L., Ryan, J .A. M., Lonczak, H. S., & Hawkins, J. D. (1998). 

Positive youth development in the United States: Research findings on evaluations of 
positive youth development programs. Retrieved May 24, 2001, from 

http://aspe.os.dhhs.gov/hsp/PostivieYouthDev99/index.htin 

Ceprano, M. A. ( 1995). Strategies and practices of individuals who tutor adult illiterates 
voluntarily. Journal of Adolescent and Adult Literacy, 59(1), 56-64. 

Chung, A. M., de Kanter, A., & Kugler, M. R. (2000, Spring). Measuring and evaluating 
child and program outcomes. School-age Review, 2^32. 

Clarke-Stewart, K. A. (1987). In search of consistencies in child care research. In D. A. Phillips 
(Ed.), Quality in child care: What does research tell us? (pp. 105-120). Washington, DC: 
National Association for the Education of Young Children. 

* Cogswell, S. H., Lochtefeld, S. S., Skaggs, A. V., Walker, J. P. (June 1998). 

Head Start’s impact on school readiness in Ohio: A case study of kindergarten students. 

Columbus, OH: Legislative Office of Education Oversight. This report is available at 

www.loeo.state.oh.us. 

Cohen, P. A., Kulik, J., & Kulik, C .C. (1982). Educational outcomes of tutoring: A meta 
analysis of findings. American Educational Research Journal, 19, 237-248. 

Condravy, J. (1995). Tutors learning about learning: An assessment. Research and Teaching in 
Developmental Education, 11(2), 43-56. 

Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues for 
field settings. Boston: Houghton Mifflin. 

Cook, T. D., & Shadish, W. R. (1994). Social experiments: Some developments over the past 
fifteen years. Annual Review of Psychology, 45, 545-581. 

Cooper, H. (1998). Synthesizing research: A guide for literature reviews (3rd edition). Applied 
social research methods series. Volume 2. Thousand Oaks, CA: SAGE Publications. 

Cost, Quality, and Outcomes Study Team. (1995). Cost, quality, and child outcomes in child 
care centers: Public report (2nd edition). Denver: CO: University of Colorado. 

Currie, J. (2001). Early childhood education programs. Journal of Economic Perspectives, 15(2), 
213-238. 




* Currie, J., & Thomas, D. (1999). Does Head Start help Hispanic children? Journal of Public 

Economics, 74(2), 235-262. 

Dryfoos, J. G. (1998). A look at community schools in 1998: Occasional paper #2 (ED423034). 
New York: National Center for Schools and Communities. 

Education Week (2002). Quality counts 2002: Building blocks for success, 27(17). 

Fashola, O. S. (1998). Review of extended-day and after-school programs and their effectiveness 
(Report No. 24). Baltimore, MD: Center for Research on the Education of Students 
Placed At Risk. 

Finn-Stevenson, M., Desimone, L., & Chung, A. (1998). Linking child care and support 
services with the school: Pilot evaluation of the school of the 21st century. 

New Haven, CN: The Yale University Bush Center in Child Development and Social 
Policy. Retrieved on October 9, 2002, from www.yale.edu/21c/resource/read/eval.html. 

* Foley, E. M., & Eddins, G. (2001). Preliminary analysis of Virtual Y After-School Program 

participants’ patterns of school attendance and academic performance: Final evaluation 
report program 1999-2000. New York, NY: National Center for Schools and 
Communities, Fordham University. 

Frede, E. (1995). The role of program quality in producing early childhood program benefits. 

The Future of Children, 5(3), 115—132. 

Fuerst, J. S., & Fuerst, D. (1993). Chicago experience with an early childhood program: The 
special case of the Child Parent Center program. Urban Education, 28, 69-96. 

* Gamel-McCormick, M., Cummings, M. E., Cornwell, J. R., & Kersteter, P. G. (2001). 

2000-2001 Head StartlECAP outcomes report: State of Delaware. 

Garber, H., & Heber, F. R. (1977). The Milwaukee Project: Indications of the effectiveness of 
early intervention and preventing mental retardation. In P. Mittler (Ed.), Research to 
practice in mental retardation, vol. 1: Care and intervention (pp. 1 19—127). Baltimore: 
University Park Press. 

Gaskins, J., & Roeger, E. (1995). Using process recording to train tutors in the writing center and 
peer responders in the classroom. Research and Teaching in Developmental Education, 
11(2), 57-69. 

* Gaynor, F. K., & Horowitz, J. E. (1998). Evaluation of the 4-H After-School Activity Program 

(ASAP) 1997-1998. San Francisco, CA: WestED. 

General Accounting Office. (September 2000). Title 1 preschool education. More children served 
but gauging effect on school readiness difficult. GAO/MEHS-00-171. 

Gilliam, W. S. (2000). On over-generalizing from overly-simplistic evaluations of complex 

social programs: In further response to Goodson, Layzer, St.Pierre and Bernstein. Early 
Childhood Research Quarterly, 15, 66-71. 

Gilliam, W. S., & Zigler, E. (2001). A critical meta-analysis of all evaluations of state-funded 
preschool from 1977 to 1998: Implications for policy, service delivery, and program 
evaluation. Early Childhood Research Quarterly, 15(4), 441-473. 



ERIC 




Graesser, A. C., Bowers, C, & Hacker, D. J. (1997). An anatomy of naturalistic tutoring. In K. 
Hogan & M. Pressley (Eds.), Scaffolding student learning: Instructional approaches and 
issues (pp. 145-183). Cambridge, MA: Brookline Books. 

Graesser, A. C., & Person, N. K. (1994). Questions asked during tutoring. American 
Educational Research Journal, 31, 104—137. 

* Gray, K., Roole, B., & AiVhitaker, G. P. (1999). How after-school programs help students do 
better. Popular Government, Summer 1999. 

Gray, S., Ramsey, B., & Klaus, R. (1983). The Early Training Project 1962-1980. In The 

Consortium for Longitudinal Studies (Ed.), As the twig is bent. Hillsdale, NJ: L. Erlbaum, 
33-70. 



Grossman, J. B., Price, M. L., Fellerath, V., Jucovy, L. C., Kotloff, L. J., Raley, R., & Walker, K. 
E. (2002, June). Multiple choices after school: Findings from the extended-service 
schools initiative. Philadelphia, PA: Public Private Ventures. 

Guralnick, M. J. (Ed.). (1997). The effectiveness of early intervention. Baltimore, MD: Paul H. 
Brookes. 

Halpem, R., & Weiss, H. (1990). Family support and education programs: Evidence from 

evaluated program experience. In Helping families grow strong: New directions in public 
policy: Papers from the Colloquium on Public Policy and Family Support (pp. 1 1 1-129). 
Chicago, n^: Family Resource Coalition. 

* Hamilton, L. S., & Klein, S .P. (1998). Achievement test score gains among participants in the 

foundations school-age enrichment program. Rand Project Memorandum (PM-858- 
EDU), September 1998. 

Head Start Bureau. (2002). Research and statistics website. Retrieved on October 9, 2002, from 

http://www.acf.hhs.gov/programs/hsb/research 

Heinsman, D. T., & Shadish, W. R. (1996). Assignment methods in experimentation: When do 
nonrandomized experiments approximate answers from randomized experiments? 
Psychological Methods, 1, 154-169. 

* Hemmeter, M. L. (2001). Kentucky preschool program. 2000 Review of research on the 

Kentucky Educational Reform Act. Retrieved from www.kier.or^preschooLhtml. 

* Henry, G. T., Gordon, C. S., Mashum, A., & Ponder, B. D. (2001). Pre-k longitudinal study: 

Findings from the 1999-2000 school year. A publication of the Applied Research Center 
at Georgia State University. Available at www.arc.gsu.edu/publications.htm#pre-k- 
reports. 

Hock, M. F. (1999). The strategic tutoring series: The MATH strategy. Lawrence, KS: 
University of Kansas, Center for Research on Learning. 

Hock, M. F., Deshler, D. D., & Schumaker, J. B. (2000). Strategic tutoring. Lawrence, KS: Edge 
Enterprises. 



O 

ERIC 



* Hock, M. F., Pulvers, K. A., Deshler, D. D., & Schumaker, J. B. (2001). The 

effects of an after-school tutoring program on the academic performance of at-risk 
students and students with LD. Remedial and Special Education, 22(3), 172—186. 

Hock, M. F., Schumaker, J. B., & Deshler, D. D. (1995). Training strategic tutors to enhance 
learner independence. Journal of Developmental Education, 19, 18-26. 

* Hoffman Clark & Associates. (July, 2001). San Diego After School Regional Consortium: 

Academic indicator report. 1999-2000. 

Hoge, R. D. & Coladario, T. (1989). Teacher-based judgments of academic achievement: A 
review of the literature. Review of Educational Research, 59, 297-313. 

Howrigan, G. A. (1988). Evaluating parent-child interaction outcomes of family support and 
education programs. In H. B. Weiss & F. H. Jacobs (Eds.), Evaluating family 
programs (pp. 95-130). New York: Aldine de Gruyter. 

* Huang, D., Gribbons, B., Kyung, S. K., Lee, C., & Baker, E. V. (2000). A decade of results: 

The impact of LA 's BEST after-school enrichment program on subsequent student 
achievement and performance. Los Angeles: UCLA Center for the Study of Evaluation. 

* Hudley, C. (1999). Problem behaviors in middle childhood: Understanding risk status and 

protective factors. Paper presented at the armual meeting of The American Educational 
Research Association in Montreal, Canada. 

* Illinois State Board of Education (2001). Illinois pre-kindergarten program for children at 

risk of academic failure: FY 2000 evaluation report. Available at 

www.isbe.net/research/pdfs/atrisk.pdf 

* Kaiser, M., & Lyons, M. (2001). Woodcraft rangers: State of California After School Learning 

and Safe Neighborhoods Partnerships Program with the Los Angeles Unified School 
District. Annual evaluation report. 1999—2000. Los Angeles, CA: Lodestar 
Management/Research, Inc. 

Karoly, L. A., Greenwood, P. W., Everingham, S. S., Hoube, J., Kilbum, M. R., Rydell, C. P., 
Sanders, M., & Chiesa, J. (1998). Investing in our children: What we know and don’t 
know about the costs and benefits of early childhood interventions. Washington, DC; 
Rand Corporation. 

Kubisch, A. C., Weiss, C. H., Schorr, L. B., & Connell, L. P. (1995). Introduction (pp. 1-21), 

In J. P. Cormell, A. C. Kubisch, L. H. Schorr, & C. H. Weiss (Eds.), New approaches to 
evaluating community initiatives: Concepts, methods, and contexts. Washington, DC; 
Aspen Institute. 

* Kurecka, P., & Claus, R. N. (July, 2000). Pre-kindergarten program product evaluation 

report, 1999-2000: An approved report of the Department of Evaluation, Testing, and 
Research, School District of Saginaw, Michigan. 

* Lamare, J. (1997). Sacramento START: An evaluation report. Sacramento, CA; Sacramento 

California Neighborhoods Planning and Development Services Department. 




49 



Lepper, M. R., Drake, M. R, & O’Donnell-Johnson, T. (1997). Scaffolding techniques of expert 
human tutors. In K. Hogan & M. Pressley (Eds.), Scaffolding student learning: 
Instructional approaches and issues (pp. 108-144). Cambridge, MA: Brookline Books. 

* LoSciuto, L., Hilbert, S., Fox, M., Porcellini, L., & Lanphear, A. (1999). A two-year evaluation 

of the Woodrock youth development project. Journal of Early Adolescence, 79(4), 
488-518. 

* Love, J. M., Kisker, E. E., Ross, C. M., Schochet, P. Z., Brooks-Gunn, J., Boiler, K., Paulsell, 

D., Fuligni, Allison S., & Berlin, L. J. (June, 2001). Building their futures: How Early 
Head Start programs are enhancing the lives of infants and toddlers in low-income 
families. Volume I: Technical report, 1-306. Available at 

http://www.acf.hhs.gov/programs/core/ongoing_research/index.html 

* Marcon, R. (2000). Educational transitions in early childhood, middle childhood, and 

early adolescence: Head Start vj'. public school pre-kindergarten graduates. 

Report presented at the Fifth National Head Start Research Conference, 

Washington DC 

Marshall, N. L., Coll, C. G., Marx, F., McCartney, K., Keife, N., and Ruh, J. (1997). After- 
school time and children’s behavioral adjustment. Merrill- Pahner Quarterly, 43(3), 
497-514. 

* Martinez, S. (2002). Four-year-old at risk program: Final evaluation. Retrieved from 

www.ksde.org/pre/atriskpreschool.doc. 

* Maryland State Department of Education. (2002). Children entering school ready to learn: 

School readiness baseline information. School year 2001- 02 by state and county. 1—14. 
Available at www.msde.state.md.us/Special_ReportsandData 

* Maxwell, K., Bryant, D., Keyes, L., & Bernier, K. (July 1997). Kindergartner’s skills in Smart 

Start counties in 1995: A baseline from which to measure change. Available at 

http://www.fpg.unc.edu/~SmartStart. 

* Maxwell, K., Bryant, D., & Miller-Johnson, S. (1999). A six-county study of the effects of 

Smart Start child care on kindergarten entry skills. The Frank Porter 
Graham Child Development Institute, University of North Carolina-Chapel Hill. 
Available at http://www.fpg.unc.edu/~SmartStart. 

McArthur, D., Stasz, S., & Zmuidzinas, M. (1990). Tutoring techniques in algebra. Cognition 
and Instruction, 7, 197-244. 

Meece, J. L. (1992). Personal social and moral development. In J. L. Meece (Ed.), Child and 
Adolescent Development (pp. 320-379). New Y ork: McGraw Hill. 

Meisels, S., Bickel, D., Nicholson, J., Xue, Y., & Atkins-Bumett, S. (2001). Trusting teachers’ 
Judgments: A validity study of a curriculum-embedded performance assessment in 
kindergarten to grade 3. American Educational Research Journal, 55(1), 73-95. 

Meisels, S. J., Jablon, J. K., Marsden, D.B., Dicmelmiller, M.L., & Dorfman, A. B. (1994). 

The Work Sampling System. Aim Arbor, MI: Rebus. 

Miller, B. M. (1995). Out-of-school time: Effects on learning in the primary grades. Wellesley, 
MA: Center for Research on Women, National Institute on Out-of-School Time. 



ERIC 



50 



Miller, B. (2001, April). The promise of after-school programs. Educational Leadership, 

55(7), 6-12. 

The National Education Goals Panel. (1997). Getting a good start in school. Washington, DC: 
National Education Goals Panel. 

National Governor's Association (2000). Issue brief: Evaluating statewide, community- 
based initiatives for young children. Washington, DC: Author. Retrieved from 

http://www.nga.org/cda/files/001221CHILDINIT.pdf. 

National Institute for Child Health and Human Development. (2001). The NICHD study of early 
child care. Retrieved from 

www.nichd.nih.gov/publications/pubs/early_child_care.htm. 

National Institute on Out-of-School Time. (2000). Measuring what matters: Meeting the high- 
stakes accountability challenge on our own terms. After School Issues, I, National 
Institute on Out-of-School Time. 

National Research Council and Institute of Medicine. (2000). From neurons to neighborhoods: 
The science of early childhood development. Committee on Integrating the Science of 
Early Childhood Development. In J. P. Shonkoff & D. A. Philllips (Eds.), Board of 
Children, Youth, and Families, Commission on Behavioral and Social Sciences and 
Education. Washington, DC: National Academy Press. 

* Northwest Regional Laboratory, Child and Family Program. (1998). An investment in 

children and families: Year 7 longitudinal study report. Can be requested from 

http://www.ocd.wa.gov/info/csd/waeceap/Longitudinal_Studies/default_longitudinal 

_study.html 

* O’Donnell, J., & Michalak, E. A. (1997). Inner-city youths helping children: After-school 

programs to promote bonding and reduce risk. Social Work in Education, 79(4), 231- 
442. 

* Oregon Department of Education. (1999). 2000 kindergarten survey report: Readiness to 

learn. Retrieved from www.ode.state.or.us/stusvc/EarlyChild. 

Peth-Pierce, R. (2000). A good beginning: Sending America’s children to school with the social 
and emotional competence they need to succeed (Monograph). Bethesda, MD: The Child 
Mental Health Foundations and Agencies Network. 

Pettit, G. S., Laird, R. D., Bates, J. E., & Dodge, K. A. (1997). Patterns of after-school care in 
middle childhood: Risk factors and developmental outcomes. Merrill- Palmer Quarterly, 
45(3), 515-538. 

* PTannestiel, J. (2001). School entry assessment project: Report of findings. Missouri 

Department of Elementary and Secondary Education 

Phillips, D. A. & Howes, C. (1987). Indicators of quality in child care: Review of research. In 
D.A. Phillips (Ed.), Quality in child care: What does research tell us? (pp. 1-19). 
Washington, DC: National Association for the Education of Y oung Children. 



* Phillips, R. S. (1999). Intervention with siblings of children with developmental disabilities 

from economically disadvantaged families. Families in Society: The Journal of 
Contemporary Services, 569-577. 

* Pierce, K. M., Hamm, J. V., & Vandell, D. L. (1999). Experiences in after-school programs 

and children’s adjustment in first-grade classrooms. Child Development, 70, 756-767. 

Posner, J. K., & Vandell, D. L. (1994). Low-income children’s after-school care: Are there 
beneficial effects of after-school programs? Child Development, 65, 440—456. 

Powell, D. (1995). Enabling young children to succeed in school. Washington, DC: American 
Educational Research Association. 

Powell-Smith, K. A, Stoner, G., Shinn, M. R., & Good IE, R. H. (2000). Parent tutoring in 
reading using literature and curriculum materials: Impact on student reading 
achievement. School Psychology Review, 29(1), 5-28. 

* Prenovost, J. (2001). A first-year evaluation of after school learning programs in four urban 

middle schools in the Santa Ana Unified School District. Doctoral dissertation. 

University of California, Irvine and Los Angeles. 

Ramey, S. L. & Ramey C. T. (1992). Early educational intervention with disadvantaged children: 
To what effect? Applied & Preventive Psychology, 1, 131-140. 

* Reisner, E. R., White, R. N., Brimingham, J., & Welsh, M. (2001). Building quality and 

supporting expansion of after -school projects: Evaluation results from the TASC after- 
school program ’s second year. Washington, DC: Policy Studies Associates, Inc. 

* Reisner, E. R., Russel, C., Welsh, M., Brimingham, J., & White, R. N. (2002). Supporting 

quality and scale in after-school services to urban youth: Evaluation of program 
implementation and student engagement in the TASC after-school program ’s third year. 

Reynolds, A. J. (2000). Success in early intervention: The Chicago Child-Parent Centers. 
Lincoln, NE: University of Nebraska Press. 

* Reynolds, A. J., Temple, J. A.. Robertson, D. L., & Mann, E. A. (2001). Long-term effects of 

an early childhood intervention on educational achievement and Juvenile arrest: A 15- 
year follow-up of low-income children in public schools. Journal of the American 
Medical Association, 255(18), 2339-2346. 

Ripple, C. H., Gilliam, W. S., Chanana, N., & Zigler, E. (1999). Will fifty cooks spoil the broth? 
The debate over entrusting Head Start to the states. American Psychologist, 54, 327-343. 

Rodriguez, E., Hirschl, T., Mead, J., and Goggin, S. (1999). Understanding the difference 4-H 
clubs make in the lives of New York youth: How 4-H contributes to positive youth 
development. Retrieved from www.cce.cornell.edu/youth/4h/news/final_reporLhtml. 

Roth, J., Brooks-Gunn, J., Murray, L., & Foster, W. (1998). Promoting healthy adolescents: 
Synthesis of youth development program evaluations. Journal of Research on 
Adolescence, 8, 423-459. 

Schinke, S., Cole, K. C., & Poulin, S. R. (1998). Evaluation of Boys and Girls’ Club of 
America ’s Educational Enhancement Program. Atlanta, GA: Author. 



O 

ERIC 



52 



* Schinke, S. R, Cole, K. C., and Poulin, S. R. (2000) Enhancing the educational achievement of 

at-risk youth. Prevention Science, /(I), 51-60. 

Schulman, K., Blank, H., & Ewen, D. (1999). Seeds of success: State pre-kindergarten 
initiatives 1998-1999. Washington, DC: Children’s Defense Fund. 

Schweinhart, L. (in press). Issues in implementing state preschool program evaluations. Special 
report on wide-scale early childhood assessment systems. Greensboro, NC: SERVE. 

Schweinhart, L. J. (1999). The three types of early childhood programs in the U.S. Paper 
presented for the National Invitational Conference on Early Childhood Learning: 
Programs for a New Age, Alexandria, Virginia, November 29-December 1, 1999. 

Schweinhart, L. J., Barnes, H. V., & Weikart, D. P. (1993). Significant benefits: The High/Scope 
Perry Preschool study through age 27 (Monographs of the High/Scope Educational 
Research Foundation No. 10). Ypsilanti, MI: High/Scope Educational Research 
Foundation. 

Scott-Little, C., Brown, G., & Dufford-Melendez, K. (December 12-14, 2001). Notes from the 
Assessing the State of State Assessment Symposium. Unpublished manuscript. 

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental 
designs for generalized causal inference. Boston: Houghton-Mifflin. 

Shadish, W. R., & Ragsdale, K. (1996). Random versus nonrandom assignment in controlled 

experiments: Do you get the same answer? Journal of Consulting & Clinical Psychology, 
64, 1290-1305. 

* Tao, F., Gamse, B., & Tarr, H. (1998). National evaluation of the Even Start family literacy 

program: 1994-1997 Final report. 

Tierney, J., Grossman, J. B., and Resch, N. (1995). Making a difference: An impact study of Big 
Borthers/Big Sisters. Philadelphia, PA: Public/Private Ventures. 

* Trousdale, D. (2000). First year evaluation of an after-school program for middle school youth. 

ERS Spectrum, Summer 2000. 

U.S. Department of Education. (2002). 21st Community Learning Centers background. 

Retrieved on October 9, 2002 from www.ed.gov/21stcclc/. 

* University of Nevada, Reno Cooperative Extension Service. (June, 1999). Reducing delinquent 

behavior and improving academic achievement in after-school programs. Retrieved 
March 2, 2001, from www.cyfernetorg. 

* Urban School Initiative. (1999). Urban School Initiative school age care project: 1998-99 

school year evaluation report. Columbus, OH: Ohio Departments of Human Services and 
Education. 

Vadasy, P. F., Jenkins, J. R., Antil, L. R., Wayne, S. K., & O’Connor, R. E. (1997). The 

effectiveness of one-to-one tutoring by community tutors for at-risk begiiming readers. 
Learning Disability Quarterly, 20, 126-137. 

Walker, K. E., Grossman, J. B., Raley, R., Fellerath, V., & Holton, G. I. (2000, December). 




53 



Extended- service schools: Putting programming in place. Philadelphia, PA: 

Public Private Ventures. 

Wasik, B. A., & Slavin, R. E. (1990, April). Preventing reading failure with one-to-one tutoring: 
A best evidence synthesis. Paper presented at the Aimual Meeting of the American 
Educational Research Association, Boston. 

Weiss, C. H. (1995). Nothing as practical as good theory: Exploring theory-based evaluation for 
comprehensive community initiatives for children and families. In J. P. Coimell, 

A. C. Kubisch, L. H. Schorr, & C. H. Weiss (Eds.), New approaches to evaluating 
community initiatives: Concepts, methods, and contexts (pp.65-92). Washington, DC: 
Aspen Institute. 

West, J., Denton, K, & Germino-Gausken, E. (February, 2000). America’s kinder gartners: 

Findings from the Early Childhood Longitudinal Study, kindergarten class of 1998-99, 
fall 1998 (NCES 2000-070). Washington, DC: National Center for Education Statistics. 

* White, R. N., Reisner, E. R., Welsh, M., & Russel, C. (2001). Patterns of student-level change 

linked to TASC participation based on TASC projects in Year 2. Washington, DC: Policy 
Studies Associates, Inc. 

* Witt, P. A., & King, T. (2001, July). Fort Worth after-school program: “A diamond in the 

rough.” College Station, TX: Texas A&M University. 

Wortman, P. M., & Bryant, F. B. (1985). School desegregation and black achievement: An 
integrative review. Sociological Methods and Research, 13{3), 289—324. 

* Xiang, Z., Schweinhart, L., Hohmaim, C., Smith, C., Storer, E., & Oden, S. (2000). Points 

of light: Third year report of the Michigan school readiness evaluation. Ypsilanti, MI: 
The High/Scope Education^ Research Foundation. 

Yoshikawa, H. (1995). Long-term effects of early childhood programs on social outcomes and 
delinquency. The Future of Children, 5(3), 51-75. 

Zaslow, M., & Halle, T. (in press). Statewide school readiness assessments: Challenges and next 
steps. Special report on wide-scale early childhood assessment systems. Greensboro, NC: 
SERVE. 

Zill, N., & West, J. (2000). Entering kindergarten: A portrait of American children when they 

begin school. Condition of Education 2000 (NCES 2000-062). Washington, DC: National 
Center for Education Statistics. 

* Zill, N., Resnick, G., McKey, R. H., Clark, C., Pai-Samant, S., Coimell, D., Vaden-Kieman, 

M., O’Brien, R., & D’Elio, M. A. (January, 2001). Head Start FACES: Longitudinal 
findings on program performance, third progress report. Available at 

www.acf.hte.gov/programs/core/ongoing_research/hs/impact_intro.html 




54 




U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education (NLE) 
Educational Resources Information Center (ERIC) 




NOTICE 

Reproduction Basis 



This documenf is covered by a signed "Reproduction Release (Blanket)" 

foim (on file wiflun the ERIC system), encompassing all or classes of 

organization and, therefore, does not require a 
Specific Document Release form. 




This document is Federally-ftmded, or carries its own permission to 
reproduce or is otherwise in the public domain and, therefore, may be 

reproduced by ERIC without a signed Reproduction Release form (either 
Specific Document" or "Blanket"). 



ERICEFF^089 (1/2003) 

o'^/3 // 



/ 



